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Executive  Summary 


The  U.S.  Army  Research  Laboratory  (ARL)  has  been  studying  urban  environments  in  response 
to  the  increased  urban  warfare  facing  the  U.S.  Armed  Forces.  Within  the  first  decade  of  2000, 
three  progressively  more  complex  urban  atmospheric  field  studies  were  conducted  at  White 
Sands  Missile  Range  (WSMR),  NM.  These  studies  examined  airflow  and  stability  around  urban 
buildings.  Concurrent  with  the  final  study,  emergency  first  response  drills  were  executed.  Based 
on  the  ARL  collaboration  with  the  local  emergency  response  professionals,  the  need  for  first 
responders  to  have  real-time  atmospheric  data  at  their  figure-tips  was  witnessed. 

ARL  responded  to  the  civilian/military  emergency  responder  need,  by  designing  the  Local-Rapid 
Evaluation  of  Atmospheric  Conditions  (L-REAC™)*  System.  From  2009  to  201 1,  ARL  went 
from  a  sketched  L-REAC™  System  concept  to  a  mature  Operational  L-REAC™  System.  The 
current  operational  system  is  an  automated,  24/7  emergency  response  decision  aid  for  airborne 
toxic  release  incidents.  The  basic  design  of  the  L-REAC™  System  includes  five  modules:  the 
Sensor,  Quality  Control  (QC),  Model,  End  User  Display  (EUD)  and  Archive  Modules.  The 
Sensor  Module  perpetually  acquires  meteorological  data  used  as  input  by  both  a  wind  field  and 
plume  model.  These  models  continually  run,  producing  near  real-time,  mapped  wind  field  and 
plume  (when  applicable)  outputs,  which  are  accessible  to  authorized  end  users.  An  Archive 
Module  saves  both  the  measured  data  and,  when  the  user  has  selected  this  option,  the  image 
output.  The  System  is  designed  to  run  attended  by  an  operator  or  without  an  operator. 

As  part  of  a  technology  transfer  process,  the  L-REAC™  System  was  subjected  to  several 
informal  and  formal  evaluations.  The  first  evaluation  examined  the  intuitive  nature  of  the  output. 
About  thirty  Gains  in  the  Education  of  Mathematics  &  Science  (GEMS)  high  school  students/ 
teachers  were  asked  to  use  the  system  to  assess  the  wind  flow  and  safe  zone  locations.  Later, 
several  Force  Protection  Exercises  and  Drills  provided  user  feedback.  In  April  2011,  the  L- 
REAC™  System  was  requested  to  provide  wind  field  projections  for  assisting  fire  fighters  in  the 
real-world  Abrams  Fire  Incident.  Training  sessions  were  subsequently  requested  by  the 
emergency  organizations.  Within  these  training  sessions,  the  first  formal  L-REAC™  System 
improvement  evaluations  were  conducted.  The  results  are  the  subject  of  this  report. 

The  evaluation  was  organized  into  two  questionnaires:  (1)  An  Administrative  Questionnaire, 
which  looked  at  the  System  from  an  overview  and  future  vision  perspective;  and  (2),  a  more 
detailed  review  of  the  System  Functionality.  The  second  questionnaire  also  served  as  an 
informal  test  for  the  students  who  had  just  completed  their  Training  I  course.  The  Functionality 
Questionnaire  was  divided  into  three  cases/scenarios:  Case  I  (No  Plume  Scenarios),  Case  II 
(Plume  Scenarios),  and  Case  III  (No  Plume/Plume  Scenarios,  in  multiple  locations). 


The  L-REAC  System  trademark  is  owned  by  the  Department  of  the  Army.  Washington  DC,  20310. 


vii 


The  results  of  the  Administrative  Questionnaire  reported  that,  from  the  remote  accessing  of 
communications  through  the  perpetually  updated  wind  and  plume  images,  the  system  was  easily 
understood  and  utilized.  Evaluators  unanimously  stated  that  they  would  recommend  the  system. 
Their  reasons  for  recommending  the  System  aligned  with  the  original  System  design  objective  of 
“saving  lives.”  The  comments  also  underscored  the  challenge  of  finding  funding  resources  to 
support  system  development  costs. 

The  individual  evaluator  applications  listed  for  the  L-REAC™  System  consisted  of  Chemical, 
Biological,  Radiological  and  Nuclear  Effects  (CBRNE)  Force  Protection  applications, 

Installation  Operations  Centers  (IOC),  Emergency  Operations  Centers  (EOC),  Fire/Rescue  and 
Emergency  Dispatch.  All  evaluators  reported  that  the  L-REAC™  System  would  be  an  asset  to 
their  profession/application. 

System  strengths  cited  included  five  comments  which  praised  the  ease  of  use,  understandability 
of  end  products,  and  the  “great”  potential  of  the  System. 

Two  scenarios  for  integrating  L-REAC™  into  the  operational  emergency  environment  were 
offered  (1)  Multiple  L-REAC™  Systems  owned  and  operated  by  various  organizations.  (2)  One 
L-REAC™  System  owned  and  operated  by  a  single  organization  serving  multiple  users 
representing  different  organizations. 

The  evaluators  offered  a  third  arrangement:  Two  (or  three)  organizations  owning  L-REAC™ 
Systems  with  one  organization  as  the  primary  operator  serving  multiple  users  of  different 
organizations.  The  second  System  would  be  installed  at  a  different  location,  and  would  serve  as 
a  backup  unit,  should  the  primary  unit/location  suddenly  lose  power  or  network  connectivity. 
Based  on  current  national  economics,  this  third  option  may  be  the  best  arrangement. 

Evaluators  unanimously  selected  the  Fire  and  Rescue  Department  (Fire  Dept)/Medical  and  the 
IOC  as  appropriate  System  owners,  with  the  majority  also  selecting  the  EOC  and  the  Incident 
Command  (IC)  on  location.  The  least  recommended  owners  were  the  installation  command 
offices.  The  top  recommendation  for  prospective  System  users  was  the  Fire  Dept/Medical,  with 
the  IOC,  EOC,  Police,  IC  on  location,  IC  in  IOC  and  Garrison  Commander  also  suggested  by  the 
majority  of  evaluators.  The  least  recommended  user  was  again  the  installation  command  office. 
The  “Dispatch”  office  was  a  written-in  suggestion  for  both  an  owner  and  user. 

The  directions  for  accessing  L-REAC™  System  output  were  unanimously  evaluated  as  “easy  to 
follow”  and  the  output  was  flagged  as  timely.  The  output  details  were  assessed  as  “enough” 
(versus  “too  many  details”).  The  Operator’s  zoom  in  capability  was  considered  very  useful  in  the 
EM  (emergency  management)  arena. 

Regarding  the  two  main  output  display  products  (current  wind  field  and  plume  overlays  mapped 
onto  a  given  area  of  interest),  the  majority  of  evaluators  considered  the  product  features  useful, 
with  78%  of  evaluators  indicating  ALL  infonnation  as  “most  useful.” 
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All  evaluator  comments  were  extremely  helpful  in  calibrating  the  value  of  the  L-REAC™ 
System  to  the  emergency  response  community.  While  funding  may  limit  future  System 
development,  the  fact  that  the  System  has  already  proven  to  have  a  positive  impact  on  both  the 
local  ARL  safety  procedures  and  the  surrounding  military/civilian  communities  indicates  that 
this  technology  has  the  potential  to  continue  making  a  constructive  difference.  And,  with  proper 
support,  the  L-REAC™  System  will  resume  being  a  candidate  for  use  by  ARL’s  primary 
customer,  the  U.S.  Army  Soldiers. 


Intentionally  Left  Blank. 
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1.  Background 


In  recent  years,  the  U.S.  military  has  seen  an  increase  in  the  number  of  combat  missions  within 
an  urban  environment.  Consequently,  the  U.S.  Army  Research  Laboratory  (ARL)  has  been 
investigating  the  atmospheric  effects  within  urban  scenarios.  In  the  early  2000s,  ARL  conducted 
three  progressively  more  complex  urban  field  studies  aimed  at  characterizing  the  airflow  and 
stability  around  a  small  cluster  of  urban  buildings.  As  part  of  the  final  Study,  three  Disaster 
Response  Drills  were  executed.  Observing  the  emergency  response  within  the  thickly 
instrumented  urban  field  site,  the  need  for  a  tool  to  link  health  and  safety  professionals  with 
relevant  atmospheric-environment  data  became  apparent.  A  survey  of  existing  technologies 
offered  potential  solutions.  However,  the  ARL  saw  a  niche  for  a  timely,  relevant,  user-friendly 
tool  that  had  not  yet  been  filled,  and  consequently,  has  been  developing  an  interactive 
information  system  to  satisfy  this  need.  This  system  is  called  the  “Local  Rapid  Evaluation  of 
Atmospheric  Conditions,”  or  “L-REAC™*”  System.  The  L-REAC™  System  was  developed 
into  an  operational  tool  between  2009  and  2011.  Once  the  system  was  exposed  to  the  “real 
world,”  the  developers  sought  a  detailed  system  evaluation  by  volunteer  and  professional 
emergency  first  responders.  The  focus  of  the  evaluation  was  to  identify  practical  system 
improvements.  This  report  documents  the  results  of  that  solicitation. 

In  this  technical  report,  section  1  summarizes  the  evolution  from  research  to  a  decision  aid 
concept,  to  an  operational  tool.  This  section  also  gives  a  quick  description  of  the  basic  and 
Operational  L-REAC™  Systems.  Section  2  describes  the  two  part  System  Improvement 
Evaluation  questionnaire.  The  evaluator  responses  to  the  questionnaires  are  reported  in  section 
3,  with  a  tally  of  all  evaluator  selections  and  comments  in  appendices  A  and  B.  Section  4  offers 
an  elaboration  and  comments  regarding  the  evaluator  responses.  Section  5  summarizes  the 
evaluation  process  and  key  findings  from  this  evaluation  exercise.  The  last  section  (6)  offers 
some  final  comments  from  both  the  author  and  the  evaluators. 

1.1  Field  Study  to  First  Responder  Tool 

In  the  early  2000s,  ARL  conducted  three  progressively  more  complex  urban  field  studies  which 
focused  on  the  airflow  and  stability  around  a  single  urban  building.  The  first  of  these  studies  was 
called  White  Sands  Missile  Range  (WSMR)  2003  Urban  Study  or  “ W03US ”  for  short.  W03US 
sought  to  verify  a  1994  Environmental  Protection  Agency  (EPA)/National  Oceanic  and 
Atmospheric  Administration  (NOAA)  wind  tunnel  study  published  by  Snyder  and  Lawson,  Jr. 
(1994),  which  identified  flow  patterns  around  various  building  sizes/dimensions.  This 
EPA/NOAA  study  also  identified  flow  features  that  appeared  to  be  repeatable  as  building 
dimensions  were  varied.  Translating  the  wind  tunnel’s  millimeter  scale  into  a  real-world  meters- 
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scale,  ARL  strategically  located  atmospheric  sensors  around  a  rectangular  office  building.  The 
sensor  data  purpose  was  to  verify  seven  airflow  features:  the  fetch  flow,  velocity  acceleration, 
velocity  deficit,  cavity  flow,  leeside  corner  eddies/vortices,  the  re-attachment  zone,  and  the 
canyon  flow.  Six  of  the  seven  features  are  displayed  in  figure  1.  The  seventh  feature  is  the 
“canyon  flow,”  which  is  an  accelerated  flow  that  occurs  between  two  parallel  buildings. 


Figure  1.  EPA/NOAA  wind  tunnel  results  show  the  airflow  pattern  around  a  single  building. 

Streamling  flow  is  from  left  to  right.  The  “canyon  flow”  is  not  shown  (Snyder  and  Lawson, 
Jr.,  1994) 

Based  on  the  successful  W03US  results,  two  subsequent  urban  studies  were  executed  around  the 
same  urban  environment,  each  with  an  increased  density  of  dynamic  and  thermodynamic 
measurements.  These  studies  were  called  WSMR  2005  Urban  Study  (W05US)  and  WSMR  2007 
Urban  Study  (W07US),  respectively.  ARL  technical  reports  documenting  these  studies  and  their 
findings  include: 

•  FIELD  STUDY  OVERVIEW:  Vaucher,  2006; 

•  DATA  PROCESSING:  Vaucher  et  al.,  2008  ( Volumes  DP-1,  DP-2,  DP-3;  2007); 

•  DATA  ANALYSIS:  Vaucher,  201 1  ( Volume  DA-1); 

•  ATMOSPHERIC  STABILITY  (Data  Analysis):  Vaucher,  2007  ( Volume  AS-1;  2007); 
Vaucher,  2008  (Volume  AS-2;  2008);  Vaucher,  2009  (Volume  AS-3;  2009). 

Concurrent  with  the  W07US  data  acquisition  were  simulated  disaster  response  drills.  These  drills 
included  a  Fire/Smoke  Release  Drill,  a  Simulated  Bomb  Threat  Drill  and  a  Simulated  Airborne 
Chemical  Release  Drill.  The  drills  involved  not  only  the  ARL  field  study  personnel,  but  the 
entire  local  ARL  workforce  and  local  Emergency  First  Responder  units.  From  the  WWI/S/Drills 
experiences,  the  need  for  a  near  real-time  atmospheric  evaluation  system  was  identified.  A  post- 
W07US  study  compared  the  drill  responses  with  the  coincident  atmospheric  conditions.  The 


2 


results  of  this  study  confirmed  the  need  for  an  emergency  first  responder  decision  aid  that  would 
provide  timely  and  relevant  atmospheric  data  (Vaucher,  2011).  Thus,  the  concept  for  the  L- 
REAC™  System  Decision  Aid  was  initiated. 

1.2  Decision  Aid  Development 

A  survey  of  atmospheric  emergency  first  responder  technologies  was  conducted,  following  the 
WOWS  (Vaucher  and  Brice,  2010).  While  several  promising  units  were  found,  the  proposed  L- 

TM  TM 

REAC  System  showed  unique  qualities.  Thus,  in  2009,  ARL  constructed  an  L-REAC  System 
Proof  of  Concept  (PoC),  using  a  Linux-Windows  dual  operating  system  (OS).  The  PoC  included 
three  core  modules  linked  by  specialized  networks.  These  modules  consisted  of  a  Sensor 
Module,  a  Model  Module,  and  an  End-User-Display  (EUD)  Module  (see  figure  2).  The 

TM 

L-REAC  System  PoC  was  documented  in  the  L-REAC™  System,  Design  and  Development, 
Volume  1  (Vaucher  et  al.,  2009). 

Local  -  Rapid  Evaluation  of  Atmospheric  Conditions  System 


ii 

Sensor  —  Model  —  End  User  Display  l?1?-090511! 

Figure  2.  The  L-REAC™  System  PoC  included  three  core  modules:  the  Sensor, 
the  Model  and  the  EUD  Modules. 

In  2010,  the  L-REAC™  System  “Prototype”  was  constructed  using  a  single  Windows  OS.  This 
“Prototype”  consisted  of  five  core  modules.  The  original  three  core  modules  were  significantly 
enhanced,  and  two  system  features  within  the  earlier  design  were  re-designated  as  full  modules. 
These  two  “new”  modules  were  labeled  the  Quality  Control  (QC)  Module  and  Archive  Module. 
The  “Prototype”  System  was  documented  in  the  L-REAC™  System,  Design  and  Development, 
Volume  2  (Vaucher  et  al.,  2010). 

TM 

In  201 1,  an  Operational  L-REAC  System  was  completed  which  continued  to  integrate  “bullet 
proofing”  design  improvements  to  each  system  module.  The  Operational  L-REAC™  System 
was  documented  in  L-REAC™  System,  Design  and  Development,  Volume  3  (Vaucher  et  al., 

TM 

2011).  One  of  the  goals  for  this  Operational  L-REAC  System  was  to  bring  the  operational 
system  into  a  “real  world”  environment,  where  professional  emergency  first  responders  could 
provide  a  practical  end  user  evaluation  of  the  product.  A  detailed  presentation  of  their  results  is 
the  goal  of  this  report. 
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Currently,  the  developers  are  working  on  a  “mobile”  version  of  the  L-REAC™  System.  This 
system,  if  funded,  will  more  tightly  manifest  some  of  the  original  concepts  for  ensuring  data 
relevancy.  In  the  next  section,  the  basic  L-REAC™  System  design  will  be  presented. 

1.3  The  Basic  L-REAC™  System  Design 

TM 

The  L-REAC  System  is  an  automated,  24/7,  emergency  response  decision  aid  designed  for 

TM 

airborne  toxic  release  incidents.  The  current  Operational  L-REAC  System  is  composed  of  five 
core  modules,  which  include  a  Sensor  Module,  QC  Module,  Model  Module,  EUD  Module  and 
an  Archive  Module.  The  Sensor  Module  is  designed  to  provide  continuous,  real-time  and 
relevant  atmospheric  data  from  a  single  and/or  an  ensemble  of  meteorological  sensors.  The  QC 
Module  allows  the  operator  to  instantly  create,  view  and  evaluate  time  series  of  all  variables 
sampled  by  the  dedicated  L-REAC™  System  sensor(s).  The  time  series  extends  from  local 
midnight  to  midnight.  The  imported  data  from  meteorological  mesonets  are  not  yet  part  of  the 
QC,  since  the  operator  has  no  control  over  their  status. 

The  Model  Module  interprets  the  matrix  of  current  meteorological  data  by  generating  a  local 
wind  field  over  an  area  of  interest  (AOI).  This  wind  field  is  continuously  updated  by  the  Sensor 
Module  data  feed  and  the  output  is  displayed  by  the  EUD  Module.  When  an  airborne  hazard 
occurs,  a  trained  operator  keys  in  the  hazard  specifications  (for  example,  hazard  type,  amount, 
release  method,  and  so  forth)  to  a  fast-processing,  emergency  response  plume  model,  which  is 
also  part  of  the  EUD  Module.  The  EUD  Module  then  automatically  assimilates  and 
synchronizes  the  wind  and  plume  model  outputs  into  both  building-  and  regional-scaled  images 
for  the  end  user  to  utilize  in  assessing  safe/hazard  zone  decisions.  The  EUD  output  is  distributed 
to  end  users  over  their  established  networks.  Updates  to  the  wind  field  (and  plume)  outputs  are 
automatically  transmitted  to  the  end  users  after  each  wind  field  model  run  is  completed.  A 
dedicated  System  communications  design  ensures  a  timely  information  flow  (on  the  order  of 
1-2  minutes  [min]  for  building  scales  and  8-10  min  for  regional  scales)  from  the  atmospheric 
sensors  and  models,  to  the  decision-maker  EUD  displays.  An  “Instantaneous  Save”  option 
allows  the  system  operator  to  zoom  in/out  of  an  end-user-specified  AOI  and  immediately 
transmit  these  images,  between  the  automated  cycles. 

TM 

The  Archive  Module  saves  the  ingested  L-REAC  data,  and  when  the  user  selects  the  option, 
saves  all  incident-EUD  imagery  as  well.  These  archive  files  can  be  used  for  incident  reviews 
and  Post-Event  data  analyses. 

1.4  The  Operational  L-REAC™  System 

The  Operational  L-REAC™  System  was  distinguished  from  the  “basic”  L-REAC™  PoC  unit  by 
the  System’s  capabilities  and  functionality.  As  the  L-REAC™  System  evolved  from  a  PoC  to  an 
operational  system,  several  automating  functions  and  code  reliability  improvements  were 
integrated.  The  goal  of  the  operational  system  was  to  minimize  the  user  technology 
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requirements  and  maximize  the  application  opportunities.  The  System  was  also  required  to  run 
reliably  over  extended  time  periods  with  and  without  an  operator,  and  to  produce  results  that 
would  satisfy  the  intended  customer  requirements. 

For  the  L-REAC™  System  developers  to  better  identify  what  the  customer  requirements  were, 
the  System  was  periodically  submitted  to  end  user  evaluations.  The  progression  of  significant 
interactions  is  summarized  in  section  2.  Each  customer  interaction  served  as  a  step  forward 
along  the  path  of  technology  transfer. 


2.  L-REAC™  System  Improvement  Evaluation  Task 


The  L-REAC™  System  technology  transfer  effort  began  by  subjecting  the  System  to  evaluations 
by  a  sample  of  selected  end  users.  In  2009,  the  intuitive  nature  of  the  L-REAC™  output  was 
assessed  by  presenting  the  mapped  output  to  teenagers  and  public  school  teachers  during  an  ARL 
sponsored  Gains  in  the  Advancement  of  Mathematics  and  Science  (GEMS)  summer  program. 

The  students  and  teachers  were  given  a  brief  tutorial  on  reading  maps  and  the  most  basic 
concepts  of  wind  flows  and  plume  projections  around  buildings.  They  were  then  asked  to 
evaluate  the  airflow  direction  by  visually  assessing  the  orientation  of  the  dedicated  L-REAC™ 
System  anemometer  mounted  on  the  classroom  building’s  roof.  Students  drew  the  results  on  a 
worksheet,  where  they  also  sketched  a  projected  “plume”  cloud  based  on  their  observed  airflow. 
Their  wind  and  plume  sketches  were  then  compared  to  the  “live”  wind  field  and  plume  model 
outputs  of  the  L-REAC™  System  simultaneously  displayed  in  the  building’s  lobby.  The  ease  at 
which  the  students  verbally  compared  their  results  to  the  “live”  L-REAC™  System  PoC  output 
confirmed  their  intuitive  understanding  of  the  system  output.  The  students  took  special  interest 
in  the  cavity  flow  feature  mapped  by  L-REAC™  System  wind  field  output.  Walking  between 
the  L-REAC™  System  lobby  display  output  and  the  parking  lot,  they  individually  verified  the 
flow  reversal.  That  is,  they  identified  the  cavity  flow  pattern  on  the  L-REAC™  System  “live” 
display  and  then  quickly  walked  out  to  the  parking  lot  to  visually  witness  the  roof  direction  via 
the  dedicated  L-REAC™  anemometer  on  the  roof,  and  simultaneously  feel  the  reversed  flow 
direction  against  their  backs.  The  student’s  ability  to  identify  and  match  the  model  output  with 
“real  world”  events  re-enforced  the  effective  interpretation  and  intuitive  nature  of  the  L-REAC™ 
System  output  design. 

From  2009-2011,  the  ARL  workforce  was  exposed  to  the  L-REAC™  System  output  during 
various  Shelter-in-Place  (SIP)  Exercises.  The  L-REAC™  System  displayed  a  “live”  output 
while  the  workforce  was  in  their  SIP  location.  As  primarily  atmospheric  scientists  and 
meteorologists,  these  peers  quickly  discerned  the  airflow  status  and  offered  instantaneous 
analysis  of  the  airflow  and  plume  outputs. 
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The  greatest  benefactors  of  the  L-REAC™  System  are  the  professional  civilian  and  military 
Emergency  First  Responders.  In  201 1  April,  the  local  area  experienced  an  extensive  fire  that 
threatened  the  entire  WSMR  base  community.  This  incident,  known  as  the  Abrams  Fire, 
escalated  to  the  point  where  serious  consideration  for  a  site-wide  evacuation  had  to  be  considered 
by  the  Commanding  General.  Early  on  during  this  multi-day  incident,  the  WSMR  Installation 
Operations  Center  (IOC)  called  on  the  L-REAC™  System  for  consultation.  Communication 
links  between  ARL  and  the  WSMR  advisors  to  the  Commanding  General  were  immediately 
authorized  for  a  live  feed.  For  the  next  three  days,  the  L-REAC™  System  produced  timely  and 
relevant  wind  field  and  sampled  data  output  for  the  WSMR  Directorate  of  Plans,  Training, 
Mobilization  and  Security  (DMPTS)/IOC,  DMPTS/Fire  and  Rescue  Department  (Fire  Dept),  and 
New  Mexico  Bureau  of  Land  Management.  Decisions  to  NOT  evacuate  the  site,  and  when  to 
implement  a  preventative  “back”  fire  were  made,  in  part,  on  the  L-REAC™  System  output. 
Post-incident  review  prompted  such  a  strong  interest  in  the  L-REAC™  System  that  training 
sessions  were  organized  for  the  voluntary  and  professional  WSMR  emergency  first  responders. 

Two  training  sessions  were  subsequently  designed  to  prepare  the  WSMR  emergency  response 
volunteers  and  professionals  to  be  L-REAC™  System  users.  Training  was  given  to 
representatives  from  three  directorates  in  ARL-WSMR,  the  DPTMS/Fire  Dept  and  the 
DTPMS/IOC.  Training  I  provided  an  interactive  learning  of  the  L-REAC™  System  output. 
Training  II  “tested”  the  Training  I  students  on  their  newly  acquired  understanding  and  doubled  as 
a  detailed  L-REAC™  System  improvements  evaluation.  The  students  were  not  graded  on  their 
answers,  so  their  responses  could  be  candid. 

Following  the  L-REAC™  System  User’s  Training,  the  WSMR  IOC  requested  additional 
instruction  to  become  L-REAC™  System  Operators.  Three  more  levels  (for  a  total  of  five 
training  levels)  of  training  were  given.  These  levels  covered  experience  with  the  wind  field 
model,  the  plume  model,  and  Operator  responsibilities  (startup,  QC,  operate,  archive  and 
shutdown).  Of  the  37  people  who  signed  in  for  the  1 1  classes  given  over  a  1 -month  period 
(FYll),  28  were  listed  as  users  and  5  as  operators.  Some  were  unable  to  complete  the  training, 
due  to  scheduling  conflicts  and  occupational  demands. 

The  real-world  L-REAC™  System  experience  of  April  2011  was  a  catalyst  for  soliciting  a 
detailed  System  analysis  from  volunteer  and  professional  emergency  first  responders.  In  the  next 
section,  the  L-REAC™  System  Improvement  Evaluation  task  objectives,  methods  and  execution 
will  be  described. 

2.1  L-REAC™  System  Improvement  Evaluation  Objectives 

The  purpose  of  the  L-REAC™  System  Improvement  Evaluation  was  to  solicit  practical  and 
visionary  improvements  from  volunteer  and  professional  emergency  responders.  The 
improvements  that  were  deemed  feasible  by  the  developers  were  to  be  immediately 
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implemented.  The  suggestions  requiring  additional  resources  or  skills  were  noted  for  future 
work. 

This  evaluation  exercise  also  served  as  an  informal  “test”  for  those  who  had  completed  the  L- 
REAC™  System  Training  I  class.  Training  I  was  a  prerequisite  for  the  evaluation  exercise.  For 
most  participants,  this  evaluation  period  was  the  first  time  they  had  independently  viewed  (from 
their  office  space)  an  Operational  L-REAC™  Output.  Although  the  scenarios  were  simulations, 
the  pace  of  visual  output  changes  was  in  keeping  with  real  world  incidents.  When  the  evaluation 
results  appear  to  reflect  a  learning  curve  of  the  evaluators,  a  note  flagging  this  potential  bias  is 
given. 

2.2  Method  of  Evaluation  (Training  II  and  Questionnaires) 

The  evaluation  project  was  designed  as  part  of  the  L-REAC™  System  training  program.  This 
choice  was  required  due  to  a  fiscally-driven  reduction  in  personnel  (for  the  L-REAC™  project), 
which  coincided  with  the  training  request  by  the  WSMR  IOC.  Consequently,  the  method  for 
evaluating  the  system  was  an  ungraded  “test”  of  understanding,  as  well  as  an  opportunity  for  the 
trainees  to  offer  detailed  feedback  on  what  they  had  learned. 

TRAINING  I:  As  mentioned  earlier,  each  trainee  was  required  to  go  through  the  Training  I: 
L-REAC™  System  Introduction  and  Demonstration.  At  the  end  of  session  I,  the  trainee  was 
given  two  evaluation  forms  and  picked  a  random  number  out  of  a  envelope.  Only  the  trainees 
were  aware  of  their  number,  and  used  this  number  for  the  top  of  their  evaluation  forms  (in  place 
of  their  name).  A  brief  survey  of  the  questions  introduced  them  to  the  questionnaire  content 
during  Training  I,  and  allowed  them  to  ask  for  clarification  regarding  any  question. 

TRAINING  II:  The  Training  II  System  Improvement  Evaluation  was  conducted  in  the  trainee’s 
own  office,  without  an  L-REAC™  System  developer  present.  Prior  to  the  start  of  the  Training  II 
session,  the  L-REAC™  Trainer  contacted  each  student  to  confirm  network  connectively  between 
their  approved  office  personal  computer  (PC)  and  the  L-REAC™  output.  At  a  pre-set  time,  the 
L-REAC™  System  was  initiated  from  the  L-REAC™  System  Operations  Room.  Following  the 
script  of  actions  explained  on  the  evaluation  form,  three  Cases  (scenarios)  were  executed. 
Students  entered  their  responses  for  each  Case.  After  the  Case  III  concluded,  the  students 
completed  a  second  questionnaire  that  focused  on  the  administrative  (bigger  picture)  angle  of  the 
L-REAC™  System.  The  following  day,  a  designated  person  collected  the  questionnaires  in  a 
large  folder.  When  the  participant’s  work  schedule  required  it,  questionnaires  were  returned 
several  days  later  and  anonymously  shuffled  into  the  large  envelope. 

Each  of  the  two  questionnaires  will  be  described  in  the  next  two  subsections. 
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2.2.1  Evaluation  of  System  Functionality 

The  first  questionnaire  filled  in  by  the  students  focused  on  the  L-REAC™  System  output 
functionality.  The  output  consisted  of  two  automatically  updating  hypertext  markup  language 
( html )  images  labeled  “1-Plot”  and  “2-Plot.”  In  the  “1-Plot”  image,  the  user  viewed  the  current 
wind  field  represented  in  vectors  and  streamlines,  overlaid  onto  a  Google  Earth  Image  (see  figure 
3).  If  a  plume  output  was  available,  this  plume  footprint  was  also  overlaid  onto  the  same  Google 
Earth  Image  (see  figure  4).  The  “2-Plot”  image  consisted  of  the  same  wind  field  and  plume 
information,  only  scaled  for  building  imagery  and  separated  into  two  mapped  images  showing  a 
wind  field  and  a  plume  display,  respectively.  When  a  plume  was  not  required,  a  message 
replaced  the  plume  image,  indicating  this  “no  known  airborne  threat”  status.  The  Plume  and  No 
Plume  versions  of  this  2-Plot  image  are  shown  in  figures  5  and  6. 


Figure  3.  One-Plot  EUD  image  without  a  Plume.  The  blue  arrows  show  the  model 

wind  field  at  2.5-meters  (m)  AGL.  The  red  arrow  indicates  a  measured  value 
at  10-m  AGL. 
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Figure  4.  One -Plot  EUD  image  with  a  Plume.  The  blue  arrows  show  the  model  wind 
field  at  2.5-m  AGL.  The  red  arrow  indicates  a  measured  value  at  10-m  AGL. 
The  Plume  includes  a  three  tiered  gradient  of  human  health  effects  overlay, 
with  each  tier  explained  on  the  display. 
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Figure  5.  Two-Plot  EUD  image  without  a  Plume.  The  wind  direction  is  represented  with 
black  arrows  on  the  wind  flow  chart.  Wind  speed  is  indicated  by  scaled  color 
gradients.  The  scale  is  below  the  plot.  The  solid  black  geometric  shapes 
represent  buildings. 
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Figure  6.  Two-Plot  EUD  image  with  a  Plume.  The  wind  flow  chart  includes  black 
arrows  which  indicate  wind  direction,  and  color  gradients  that  indicate  the 
wind  speed.  The  plume  overlay  uses  the  same  3-tiered  system  displayed  on 
the  1-Plot,  with  the  tiers  explained  below  the  Toxic  Plume  image. 

The  System  Functionality  Questionnaire  was  subdivided  into  three  Cases,  each  representing 
foundational  functions  for  the  L-REAC™  System  output.  These  Cases  were: 

Case  I:  No  Plume  Scenarios: 

•  STATUS:  Wind  Field  Model  and  No  Plume. 

•  SCENARIOS:  Entire  AOI,  three  Mesonet  Locations,  Medical  Facility,  a  User  Selection. 

Case  II:  Plume  Scenarios: 

•  STATUS:  Wind  Field  Model  and  Pre-determined  Plume. 

•  SCENARIOS:  Training  site-large  image  (1)  and  zoomed-in  images  (3). 

Case  III:  No  Plume/Plume  Scenarios,  with  multiple  locations: 

•  STATUS:  Wind  Field  Model  with  No  Plume  &  Plume  scenarios. 

•  SCENARIOS:  Training  site-No  Plume  &  Plume;  Plume  at  user-selected  site  within  AOI. 

In  Case  I,  the  trainees  were  able  to  orient  themselves  using  the  first  wind  field  image,  which 
covered  the  entire  AOI.  Each  scenario  step  was  updated  twice,  to  ensure  that  the  trainees  had 
enough  time  (about  4-5  min)  for  entering  evaluation  feedback  onto  the  forms.  After  the  entire 
AOI  was  viewed  (twice),  the  L-REAC™  System  Operator  went  to  the  meteorological  mesonet 
locations  listed,  enabling  the  students  to  assess  measured  (red)  and  modeled  (blue)  wind  vectors. 
The  Medical  Facility  was  used  as  a  familiar  site  to  most  trainees.  The  “user  selection”  of  Case  I, 
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allowed  one  of  the  trainees  to  select  any  site  for  the  L-REAC™  System  Operator  to  locate  within 
the  AOI.  This  action  simulated  the  challenge  of  describing  an  incident  site  over  a 
communication  line,  while  not  being  able  to  physically  point  to  it  on  a  map. 

Case  II  added  the  plume  output  to  the  wind  field,  using  a  source  point  in  the  vicinity  of  the 
training  site.  The  trainee  group  then  evaluated  various  overview  (spatial)  scales  provided  by  the 
L-REAC™  System  Operator. 

Case  III  started  with  only  a  wind  field  at  the  training  site.  The  removal  of  the  plume  flagged  the 
start  of  Case  III.  The  plume  was  subsequently  added.  A  trainee  (volunteered  in  advance  of  the 
session)  was  than  tasked  to  describe  “the  correct”  location  for  the  plume.  Again,  a  secondary 
goal  of  this  exercise  was  to  give  trainees  the  opportunity  to  experience  effective  field-to-operator 
communications  while  under  a  simulated  deadline.  Finally,  this  same  trainee  selected  a 
completely  different  location  for  the  wind  field  and  plume.  The  need  to  communicate  efficiently 
and  effectively,  and  the  need  for  the  other  end  users  to  wait  patiently  for  this  information  to  be 
delivered  and  processed,  was  the  main  educational  point  of  this  last  task  of  Case  III. 

The  System  Improvement  Evaluation  exercise  was  concluded  when  the  main  images  of  the 
1-Plot  and  2-Plot  were  returned  to  the  statement,  “The  L-REAC™  System  is  currently  off-line.” 
At  this  point,  the  evaluator/trainee  proceeded  to  the  second  questionnaire,  which  is  described  in 
the  next  section. 

2.2.2  Evaluation:  Administrative  Questions 

The  second  questionnaire  focused  on  administrative  (“big  picture”)  questions  and  applications. 
The  first  elements  evaluated  were  the  instructions  for  accessing  the  L-REAC™  System,  and  the 
L-REAC™  System  Output  with  respect  to  the  evaluator’s  professional  requirements.  A  short 
prose  (written)  section  allowed  them  to  describe  their  overall  impression,  where  they  thought  the 
L-REAC™  product  future  vision  should  be,  and  to  make  marketing  suggestions.  The 
questionnaire  concluded  by  soliciting  “Final  Comments.” 

2.3  Tabulating  Results 

Once  the  questionnaires  were  collected,  a  “key”  was  created  on  which  the  “filled-in-the-blank” 
responses  were  collated  and  the  evaluator  comments  were  transcribed.  Tallies  were  converted 
into  percentages  based  on  the  total  number  of  forms  completed.  Appendices  A  (Functionality 
Questionnaire)  and  B  (Administrative  Questionnaire)  present  the  tally  of  all  evaluator  selections 
and  comments.  For  the  three  Cases  System  Functionality  questionnaires,  ten  forms  were 
completed.  For  the  Administrative  Questionnaire,  nine  forms  were  completed. 

Some  of  the  evaluators  skipped  questions.  In  these  situations,  the  percentage  was  still  based  on 
the  total  questionnaires  filled  out.  However,  a  note  was  logged  with  the  response  indicating  that 
the  actual  percentages  may  actually  be  higher. 
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A  series  of  questions  examining  the  ease  of  entering  plume  model  information  was  included  on 
the  evaluation  forms.  Since  the  Training  II  level  did  not  include  entering  plume  model 
information,  these  questions  were  ignored. 


3.  System  Improvement  Evaluation  Results 


At  the  ARL  “After  Action  Review”  for  the  2011  April  Abrams  Fire  Incident,  a  direct  line  of 
communication  between  the  L-REAC™  System  developers  and  the  WSMR  DPTMS/Fire  Dept 
and  DPTMS/IOC  was  established.  As  part  of  this  direct  feedback,  several  details  for  system 
output  improvements  were  defined,  tasked  for  action,  and  then  implemented. 

In  the  next  three  sections,  a  sample  of  the  implemented  System  improvements  from  the  After 
Action  Review  is  given,  along  with  a  summary  of  the  administrative  and  more  detailed, 
functionality  evaluation  questionnaire  results. 

3.1  Implemented  Evaluation  Results 

As  mentioned  earlier,  the  Abrams  Fire  Incident  allowed  WSMR  Fire  Fighters  direct  access  to  the 
L-REAC™  System  output  while  executing  their  professional  tasks.  Consequently,  the  gaps  in 
Fire  Fighter  requirements  and  the  System  design  were  easily  discerned.  One  such  gap  was  a 
need  to  access  specific  meteorological  measurements  for  completing  calculations  that  assessed 
“back  burns”  and  fire  status  conditions.  These  calculations  require  specific  units  of  measure. 
Consequently,  the  L-REAC™  System  was  re-designed  to  include  a  list  of  the  available,  relevant 
dedicated,  and  mesonet  meteorological  data  with  the  1-Plot  image.  The  units  of  measure 
selected  were  based  on  those  units  requested.  The  time -relevancy  of  the  measurement  was 
defined  as  no  more  than  31  min  from  the  current  time.  The  spatial-relevancy  of  the  data  (how 
close  the  measurement  was  to  a  given  incident)  was  left  to  the  Fire  Fighter’s  discernment,  as  all 
mesonet  stations  were  included  in  the  list  displayed  on  the  EUD  1-Plot  output.  This  decision  by 
the  designers  accommodated  those  potential  incidents  that  may  extend  to  multiple  sites. 

Note:  If  no  mesonet  data  are  available,  the  wind  field  model  is  still  able  to  function, 
underscoring  the  value  of  having  a  dedicated  sensor  as  part  of  the  original  L-REAC™  System 
design. 

A  second  practical  suggestion  by  the  First  Responders  came  as  a  consequence  of  viewing  the 
EUD  output  under  brilliant  sunlight.  Computer  screens  are  already  difficult  to  read  under  normal 
sunlight.  For  the  L-REAC™  System  output,  we  needed  to  intensify  color  contrasts,  enabling  the 
end  user  to  issue  a  quick  glance  at  the  current  wind  flow  and  plume  patterns  to  glean  an 
immediate  discernment  of  application.  The  primary  target  for  this  improvement  was  the  plume 
model  output. 
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The  current  plume  model  creates  a  three-tiered  gradient  for  plume  concentrations,  as  well  as  an 
uncertainty  curve.  On  paper,  these  colors  were  sometimes  difficult  to  view  when  overlaid  onto  a 
detailed  satellite  map.  In  the  field,  especially  over  a  desert  terrain,  the  yellows  and  oranges,  in 
particular,  can  be  even  more  difficult  to  discern.  Thus,  the  L-REAC™  designers  set  out  to 
improve  the  color  contrasts.  The  suggested  software  “fix”  was  conveyed  to  the  plume  model 
developers,  who  acknowledged  the  task  by  issuing  a  code  improvement  and  informed  us  that 
their  next  plume  model  release  will  also  include  this  enhancement. 

A  third  practical  suggestion  came  from  the  System  Operators  (who  were  also  the  developers) 
responding  to  a  request  to  extend  the  system  applications  to  a  larger  AOI.  The  scaling  of  the 
5-m  high  resolution  model  was  sufficient  for  building  environments  in  a  local  community. 
However,  when  the  threat  was  in  the  surrounding  mountains  (such  as  a  wild  fire),  this  scale 
needed  to  be  expanded.  A  100-m  resolution  wind  model  was  already  within  to  the  L-REAC™ 
System  design.  However,  since  the  length  of  processing  time  exceeded  the  1-2-min  period,  this 
model  scale  had  not  been  included  in  the  automated  scripts.  Based  on  the  end  user  feedback,  the 
8-10-min  wait  for  this  output  was  considered  reasonable  when  balanced  with  the  application. 
Consequently,  the  L-REAC™  System  was  redesigned  to  include  three  scales  within  the 
automated  operational  system.  These  scales  have  been  labeled  “building”  (horizontal  AOI  is 
about  2  km  x  2  km),  “cantonment”  (AOI  is  about  25  km  x  25  km)  and  “regional”  scales  (AOI  is 
about  50  km  x  50  km). 

A  fourth  System  improvement  came  after  implementing  the  three  model  scales  into  the 
automated  operational  system.  Even  though  the  users  insisted  that  8-10  min  was  worth  waiting 
for  (they  made  this  assessment  while  sitting  in  a  conference  room),  the  high  pressure  reality  of 
the  first  responder  environment  seemed  to  prove  otherwise.  Consequently,  the  L-REAC™ 
designers  devised  an  “Instant  Save”  button  for  the  L-REAC™  System  Operator.  The  function  of 
this  iconic  button  is  to  save  whatever  is  currently  being  displayed  for  the  AOI  map,  winds  and/or 
plume  outputs.  The  intended  purpose  of  this  feature  is  to  allow  the  operator  to  send  “new” 
imagery  while  a  slower  processing  regional  scale  model  is  running.  The  “new”  imagery  could  be 
an  expanded  or  “zoomed  in”  view  of  the  existing  wind  field  over  a  given  AOI.  Or,  if  no  model 
data  are  available  (such  as  during  the  start  up  or  switching  locations),  the  Operator  can  still  send 
an  image  of  the  requested  AOI,  to  confirm  that  the  incident  location  and  image  perspective  are 
correct  and  optimal  for  the  user’s  application. 

3.2  Administrative  Questionnaire — Overall  System  Evaluation 

The  Administrative  Questionnaire  allowed  the  voluntary  and  professional  emergency  first 
responders  to  evaluate  (1)  their  overall,  initial  impression  of  the  L-REAC™  System,  (2)  where 
they  thought  the  L-REAC™  System  might  best  be  integrated  into  the  existing  first  response 
network  of  users,  and  (3)  to  assess  end  product  details  that  were  most  relevant  to  their 
applications.  Each  area  will  be  addressed  in  a  separate  subsection. 
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Note:  Evaluator  #24  did  not  fill  out  this  questionnaire  form,  making  the  total  number  of 
persons  completing  this  form  equal  to  nine.  Also,  Appendices  A  (Functionality 
Questionnaire)  and  B  (Administrative  Questionnaire)  present  the  tally  of  all  evaluator 
selections  and  comments. 

3.2.1  Overall  Impression  of  the  L-REAC™  System 

Evaluators  were  asked:  Would  you  recommend  this  system  be  purchased?  If  so,  why? 

All  evaluators  who  responded  to  this  question  answered  in  the  affirmative!  (#11  missed  this 
question).  Their  explanations  for  “why?”  included: 

•  It  can  save  lives!  (#16) 

•  It  would  be  useful  with  local  &  county  dispatch/fire  &  EOC  [Emergency  Operations 
Center]  ’s.  (#10) 

•  This  tool  would  be  helpful  for  emergency  response  and  building  evacuations  all  over  post. 
(#13) 

•  It  provides  real-time  meteorological  information  to  the  end  user  allowing  him/her  to  make 
timely  and  important  decisions.  (#14) 

•  It  has  virtually  every  bit  of  information  first  responders  need.  (#8) 

•  Useful  tool  in  determining  any  wind  plume  threats.  (#3) 

•  This  tool ’s  ability  to  determine  wind  flow  around  structures  is  very  useful  for  Fire  Fighters 
and  EM  (emergency  management).  Selling  the  product  would  be  great;  too  bad  those  that 
need  it  most  would  most  likely  not  afford  it.  (#18) 

Evaluator  #18’s  observation  that  those  that  need  it  most  would  most  likely  not  afford  it,  has, 
unfortunately,  proven  correct.  However,  we  were  encouraged  by  the  subsequent  findings  that 
elaborated  on  the  above  favorable  responses. 

The  evaluators  were  asked  to  describe  the  application(s)  for  which  they  would  use  the  L- 
REAC™  System.  Their  responses,  in  numerical  order,  were:  L-REAC™  application  is  (for). . . 

•  Ops  ( Operations )  Center  and  CBRNE  ( Chemical ,  Biological,  Radiological  and  Nuclear 
Effects).  (#8) 

•  For  all  of  the  above  (Fire/Rescue,  Dispatch)  plus  for  emergency  use  such  as  plume  data. 

(#10) 

•  IOC.  (#11) 

•  Force  Protection/personnel  notification/evacuation.  (#13) 

•  CBRNE  applications  for  displaying  plume  modeling  for  decision  makers.  (#14) 
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•  IOC/EOC.  (#15) 

•  Emergency  MST  (management)  which  encompasses  Fire/Police/Dispatch  and  providing 
info  for  decision  makers.  (#18) 

All  responding  evaluators  reported  that  “yes,”  the  L-REAC™  would  help  them  do  their  work 
(#18  missed  this  “yes”/“no”  question). 

What  the  evaluators  liked  about  the  system  included: 

•  A  great  tool  with  potential.  (#1 1) 

•  Very  easy  to  understand  and  read  winds.  (#15) 

•  Clear  and  concise  output  and  (user  friendly).  (#16) 

•  Easy  to  use  &  understand.  It  helped  with  the  initial  overview  done...  a  week  prior.  (#10) 

•  Clear  and  concise.  Displays  the  most  pertinent  information.  (#8) 

•  Everything:  wind  flow,  speed,  current  data,  (and)  toxic  plume  data.  (#18) 

Noteworthy  attributes  cited  were: 

•  Real  time  data  and  visual  with  buildings  and  surrounding  area.  (#3) 

•  It  gives  reed  time  information  and  can  also  produce  a  plume  (if  necessary).  (#14) 

•  The  capabilities  for  reed  world  use  as  a  force  protection  tool.  (#13) 

The  System  strengths  recognized  by  the  evaluators  were: 

•  Ease  of  use.  (#11  and  #14) 

•  It  gives  enough  information  so  that  if  someone  outside  the  organization  came  to  assist,  they 
should  be  able  to  assess  the  situation  relatively  quickly.  (#8) 

•  The  amount  of  different  data  elements.  (#16) 

•  Wind  speed,  direction  and  flow.  (#18) 

•  Wind  speed  and  direction — the  probability  of  the  plume  direction,  and  safe  points.  (#3) 

•  The  ability  to  add  a  plume.  (#10  and  #15) 

•  The  ability  to  determine  hazards  plume  areas  for  personnel  evacuation  purposes.  (#13) 

3.2.2  Integrating  L-REAC™  into  the  Emergency  Response  Community 

The  evaluators  were  given  two  choices  for  how  the  L-REAC™  System  might  be  integrated  into 
an  existing  community’s  Emergency  Operations.  Using  the  context  of  a  Government  Range 
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(such  as  WSMR),  one  option  was  that  multiple  organizations  would  each  have  their  own 
dedicated  L-REAC™  System.  The  second  option  was  that  one  organization  would  have  a 
dedicated  System  with  multiple  organizations  viewing  the  output  (current  design).  Their 
responses  showed  that  56%  favored  multiple  organizations  with  each  having  a  dedicated 
L-REAC™,  33%  chose  one  organization  running  the  L-REAC™  for  multiple  organizations,  and 
one  evaluator  did  not  answer  the  question. 

Those  who  favored  multiple  organizations  explained  their  choice  with  the  following: 

•  Maybe  2  or  3  organizations,  in  the  case  that  1  organization  went  down,  no  electricity,  etc. 

(#3) 

•  It  would  be  good  to  have  at  least  another  organization  with  the  system,  in  case  the  initial 
L-REAC™  System  is  down  or  destroyed.  (#10) 

•  Can  conceptualize  the  situation  fast.  (#11) 

•  Each  organization  would  be  able  to  adapt  for  specific  mission  requirements.  (#13) 

•  Everyone  would  have  different  uses.  (#15) 

•  WSMR  IOC  operates  24/7  and  having  on  hand  availability  is  important.  No  need  for  ARL 
operators  to  arrive.  (#18) 

The  evaluators  who  preferred  a  single  organization  with  multiple  users  offered  these 
explanations: 

•  It  would  be  best  for  different  organizations  to  have  a  common  point  of  reference.  (#8) 

•  This  would  cdlow  for  one-source  information  dissemination  to  multiple  agencies  and  avoid 
confusion  and  distortion  of  information.  (#14) 

•  Better  uniformity  and  control  of  data.  (#16) 

Presuming  that  multiple  organizations  could  have  their  own  dedicated  L-REAC™  System,  the 
evaluators  were  asked  to  select  a  variety  of  potential  organizations  to  own  a  System,  and/or  add 
their  own  selection.  Table  1  shows  the  results  of  the  recommended  organizations: 


Table  1.  Multiple  organizations  owning  an  L-REAC™  System. 


Organization 

IOC 

Fire/ 

Medical 

EOC 

Police 

*IC: 

in 

IOC 

AIC:  on 
location 

Garrison 

Commander 

Commanding 

General 

+Other: 

Dispatch 

%of 

evaluators 
selecting  this 
option 

100% 

(9) 

100% 

(9) 

78% 

(7) 

4% 

(4) 

44% 

(4) 

56% 

(5) 

33% 

(3) 

22% 

(2) 

11% 

0) 

*IC:  in  IOC  =  Incident  Command  in  the  IOC. 

*IC:  on  location  =  Incident  Command  at  the  location  of  the  incident. 
+User-entered  selection  was:  WSMR  dispatch  (#18) 
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Presuming  that  the  L-REAC™  System  was  run  in  an  IOC,  the  evaluators  were  asked  to  select,  to 
what  groups  or  professionals  would  they  recommend  the  L-REAC™  System  output  be  sent? 
Table  2  shows  the  results  for  organizations  that  should  receive  output: 


Table  2.  One  IOC  running  L-REAC™  System. 


Organization 

IOC 

Fire/ 

Medical 

EOC 

Police 

*IC: 

in 

IOC 

AIC:  on 
location 

Garrison 

Commander 

Commanding 

General 

+Other: 

%  of 

evaluators 
selecting  this 
option 

89% 

(8) 

100% 

(9) 

89% 

(8) 

89% 

(8) 

67% 

(6) 

78% 

(7) 

56% 

(5) 

33% 

(3) 

22% 

(2) 

*IC:  in  IOC  =  Incident  Command  in  the  IOC. 

*IC:  on  location  =  Incident  Command  at  the  location  of  the  incident. 
+OTHER: 


•  Senior  commanders  tend  to  overreact  with  too  much  info{ rmation;)  Best  to  keep  the  info( romation)  at  the  responder 
and  dispatcher  Ievel(s).  Also,  would  like  DES  911  center  added.  (#18) 

•  CBRNE  (Operations)  as  well.  (#8) 


3.2.3  Applying  the  L-REAC™  System 

The  instructions  for  accessing  the  L-REAC™  output  from  a  remote  location  were  unanimously 
assessed  as  “easy  to  follow.”  The  L-REAC™  System  output  was  assessed  as  being  timely,  with 
the  following  comments  added: 

•  Good  to  have  live  data  when  presented  questions  from  the  IOC.  (#3) 

•  Yes;  but  during  an  emergency  it  would  be  nice  to  have  update  quicker  when  there  was  a 
plume  involved.  (#10) 

•  But  I  think  that  we  could  lessen  the  update  times  (if  possible).  (#14) 

The  level  of  details  provided  in  L-REAC™  System  output  was  evaluated  as  “enough”  (versus 
“too  many  details”).  The  subsequent  comments  were  provided: 

•  Yes;  it  might  be  more  helpful  if  there  were  more  known  (RED)  arrows  in  the  area.(#  10)* 

•  Enough  detail  for  now  I  use  it  and  a  large  map  to  view  immediate  then  potential  effects. 

(#11) 

•  From  my  perspective  the  system  has  the  required  details.  (#13) 

•  It  is  providing  what  is  needed  for  decision  makers  to  get  timely  and  up-to-date  data  to  deal 
with  the  crisis  at  hand.  (#14) 

•  Perhaps  the  type  of  chemical  and  the  N-hour  could  be  included.  (#8) 

•  Enough  details.  (#16  and  #18) 
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*Note:  The  red  arrows  displayed  are  measured  meteorological  values  (versus  the  blue  model¬ 
generated  wind  vectors).  The  red-arrowed  data  are  ingested  from  a  mesonet  of  sensors,  which  is 
outside  of  the  L-REAC™  System  control. 

A  zoom-in  and  -out  capability  was  integrated  into  the  evaluation  process,  as  a  secondary  feature. 
Some  of  the  comments  regarding  this  capability  were: 

•  The  zoom  in/out  feature  Focuses  in  the  area  of  interest.  (#3) 

•  This  makes  it  easier  to  be  sure  where  on  WSMR  the  incident  is  located  for  those  unfamiliar. 

(#8) 

•  Yes,  very  useful  in  the  emergency  mgt  (management)  arena.  (#18) 

The  details  within  the  1-Plot  and  2-Plot  EUD  outputs  were  assessed,  with  respect  to  the 
evaluator’s  applications.  The  results  below  indicate  the  percentage  of  evaluators  which 
considered  the  plot  features  useful: 

Note:  Figures  3  through  6  display  a  sample  EUD  output  with  the  details  that  the  evaluators  were 
assessing. 

1-plot  graphic: 

100%  An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

100%  The  data  time  stamp. 

100%  The  height  of  output  winds  presented. 

100%  The  wind  field  model  wind  arrows  (blue). 

100%  The  wind  field  model  streamlines  (yellow). 

100%  The  measured  wind  arrows  (red). 

89%  A  “No  Plume”  notice. 

100%  A  plume  overlay. 

100%  A  plume  gradient  description  of  the  gradients  used. 

100%  The  Surface  Automated  Meteorology  System  (SAMS)  data  list. 


2-plot  graphic: 

100%  Two  area  maps. 

100%  Time  stamp  of  wind  data. 

100%  The  height  of  output  winds  (Left  plot  header). 

100%  The  wind  model  streamlines  (Left  plot). 

100%  The  color- gradient  wind  speeds  (Left  plot). 

89%  A  “No  Plume”  notice. 

100%  A  plume  overlay  (Right  plot). 

100%  A  plume  gradient  description  (Right  plot). 

After  the  evaluator  selected  which  feature  was  useful,  they  were  asked  which  feature(s)  were 
MOST  useful  to  their  application.  Seventy-eight  percent  of  the  evaluators  (7)  indicated  that  ALL 
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the  information  was  “most  useful.”  Evaluator  #1 1  did  not  answer  the  question.  And,  evaluator 
#18  selected  the  following  as  “most  useful”  items: 

1 - plot  graphic:  (#18) 

•  An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

•  The  wind  field  model  wind  arrows  (blue). 

•  The  wind  field  model  streamlines  (yellow). 

•  The  measured  wind  arrows  (red). 

•  A  plume  overlay. 

2- plot  graphic:  (#18) 

•  Two  area  maps. 

•  The  wind  model  streamlines  (Left  plot). 

•  A  plume  overlay  (Right  plot). 

The  Administrative  Questionnaire  included  two  sections  entitled  “Future  Vision”  and  “Final 
Comments.”  The  responses  from  these  questions  will  be  given  in  the  final  subsection  of  this 
section. 

3.3  System  Functionality  Questionnaire  -  Detailed  Review  of  Product 

The  L-REAC™  System  functionality  was  examined  through  three  scenarios,  as  explained  in  an 
earlier  section.  For  summary  purposes,  each  scenario  will  be  given  the  following  labels  based  on 
their  primary  focus: 

•  Case  I:  No  Plume  Scenario. 

•  Case  II:  Plume  Scenario. 

•  Case  III:  No  Plume/Plume  Scenario. 

3.3.1  Evaluating  the  L-REAC™  System  Communication  Design 

An  evaluation  of  the  communication  design  was  performed.  This  network  design  linked  the 
remotely  located  evaluator  with  the  L-REAC™  System  output.  All  but  one  person  reported  that 
the  connection  ran  without  any  interruptions.  The  single  exception  occurred  during  Case  II,  with 
no  description  of  the  interruption  provided.  Table  3  summarizes  the  results  from  System 
Functionality  question  #1.1. 
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Table  3.  System  Functionality  Question  #1.1. 


1.1  Experience  with  L-REAC™  output  access. 

Case  I: 

No  Plume 

Case  II: 

Plume 

Case  III: 

No  Plume/Plume 

(a)  DMZ  connection  ran  without  any  interruptions. 

90%  (9) 

70%  (7) 

80%  (8) 

(b)  DMZ  connection  ran  with  interruptions.  (Describe) 

0%  (0) 

10%  (1) 

0%  (0) 

(c)  Other 

0%  (0) 

0%  (0) 

0%  (0) 

Total  Evaluators: 

9** 

g>fC5lC++ 

**Total  Evaluators  (a,  b,  c)  ->  #24  missed  this  question  1.1  for  Cases  I,  II,  III. 
++Total  Evaluators  (a,  b,  c)  ->  #8  missed  this  question  1.1  for  Cases  II  and  III. 


The  frequency  of  output  updates  was  determined  to  be  sufficient  by  the  majority  of  evaluators 
(refer  to  table  4). 

Table  4.  System  Functionality  Question  #1.2. 


1.2  Frequency  of  output  updates. 

Case  I: 

No  Plume 

Case  II: 

Plume 

Case  III: 

No  Plume/Plume 

(a)  Frequency  of  output  updates  was  sufficient. 

70%  (7)* 

60%  (6)+ 

50%  (5) 

(b)  Frequency  of  output  updates  needs  to  be  more  often. 

10%  (1) 

20%  (2) 

30%  (3)A 

(c)  Frequency  of  output  updates  can  be  less  often. 

0%  (0) 

0%  (0) 

0%  (0) 

(d)  Other 

10%  (1)* 

0% 

0% 

Total  Evaluators: 

9** 

g^:^:++ 

**Total  Evaluators  (a,  b,  c)  ->  #24  missed  this  question  1.2  for  Cases  I,  II,  III. 
+++Total  Evaluators  (a,  b,  c)  ->  #8  missed  this  question  1.2  for  Cases  II  and  III. 


Comments  (1.2):  Frequency  of  output  updates 

*Case  I:  (a)  For  this  case.  (#10) 

(d)  Was  sufficient;  temporarily  dropped  out.  (#3) 

+Case  II:  (a)  With  use  of  the  F5  key  every  now  and  then.  (#10) 

ACase  III:  (b)  Might  need  quicker  updates  in  dealing  with  a  plume.  (#10) 


3.3.2  Basic  Feature  Recognition  on  the  1-Plot  and  2-Plot  EUD  Outputs 

Evaluators  established  a  baseline  of  understanding  for  the  L-REAC™  System  Output  by 
identifying  up  to  10  items  on  the  1-plot  EUD.  For  the  No  Plume  scenario  (Case  I),  all  evaluators 
were  able  to  locate  the  AOI  map,  date/time  stamp,  wind  height,  model  wind  arrows  and  model 
streamlines.  The  measured  wind  vector  and  “SAMS”  (mesonet)  data  list  were  also  recognized 
by  the  majority  of  evaluators. 

When  the  plume  was  added  to  the  scenario  (Case  II,  1-Plot),  all  evaluators  located  the  AOI  map, 
date/time  stamp,  plume  notice,  and  plume  gradient  description.  Nearly  all  evaluators  identified 
the  model  wind  arrows,  model  streamlines,  and  mesonet/SAMS  data  list  on  the  1-Plot.  The  two 
features  that  challenged  the  evaluators  were:  wind  height  and  the  measured  (red)  wind  arrows. 
This  response  is  understandable,  since  the  wind  height  is  implied  from  the  2-plot  (as  taught  in  the 
training  classes)  and  the  measured  wind  arrows  are  limited  to  their  sparse  distribution.  These 
sampled  (red)  wind  vectors  may  also  not  be  in  view  to  the  end  user,  if  the  L-REAC™  System 
Operator  narrows  (zooms  in)  the  field-of-view  for  the  user’s  application.  For  either  output  type 
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(“in  view”  or  “out  of  view”),  all  time -relevant  and  available  mesonet  data  are  listed  on  the  1-Plot 
with  each  update. 

Case  III  (1-Plot)  included  both  plume  and  no  plume  scenarios.  All  evaluators  identified  the  AOI 
map.  The  majority  of  evaluators  located  the  remaining  output  features.  Table  5  tallies  the  results 
from  System  Functionality  question  #2.1  (1-Plot  Graphic). 

Table  5.  System  Functionality  Question  #2.1  (1 -Plot  Graphic). 


2.1  During  the  evaluation  period,  which  of  the  following  was 
seen  on  your  output  display?  1-Plot  Graphic 

Case  I: 

No  Plume 

Case  II: 

Plume 

Case  III: 

No  Plume/Plume 

(a)  An  area  of  Interest  map  (Google  Earth  Map) 

100%  (10) 

100%  (10) 

100%  (10) 

(b)  The  date  and  time  stamp. 

100%  (10) 

100%  (10) 

90%  (9) 

(c)  The  height  of  the  output  winds. 

100%  (10) 

70%  (7)  + 

80%  (8) 

(d)  The  wind  field  model  wind  arrows  (blue). 

100%  (10) 

90%  (9) 

90%  (9) 

(e)  The  wind  field  model  streamlines  (yellow). 

100%  (10) 

90%  (9) 

90%  (9) 

(f)  The  measured  wind  arrows  (red). 

90%  (9) 

70%  (7) 

70%  (7)A 

(g)  A  “No  Plume”  notice.  [If  not  relevant,  put  N/A  on  line.] 

90%  (0) 

100%  (10) 

80%  (8) 

(h)  A  plume  overlay. 

N/A 

N/A 

70%  (7) 

(i)  A  plume  gradient  description  of  the  gradients  used. 

N/A 

100%  (10) 

70%  (7) 

(j)  The  WSMR  Surface  Automated  Meteorology  System  (SAMS) 
data  list  (text  box). 

90%  (9) 

90%  (9) 

90%  (9) 

Comments:  2.1  L-REAC™  System  Output  {1 -plot graphic) 

+Case  II:  (c)  I  missed  the  red  arrow  on  the  first  screen,  did  not  see  it  on  the  second  zoomed  in  screen.  (#10) 
ACase  III:  (f)  Barely.  (#18) 


For  the  2-Plot  Graphic  (see  table  6),  the  evaluator  response  indicated  an  improved 
comprehension  between  Case  I  (No  Plume)  and  Case  II  (Plume)  for  the  “two  area  maps”  and  “a 
plume  gradient  description.”  All  evaluators  who  answered  the  questions  found  the  wind  data 
time  stamp,  wind  height,  wind  model  streamlines,  and  color- gradient  wind  velocities. 

Case  III  (No  Plume/Plume,  2-Plot)  had  high  scores  for  all  the  elements,  especially  the  wind  data 
time  stamp,  wind  height,  wind  model  streamlines  and  color- gradient  wind  velocities.  Table  6 
tabulates  the  results.  Note  that  there  were  some  questions  regarding  evaluator  #16  entries;  which 
may  have  been  a  function  of  the  legibility  of  the  written  responses.  Each  Case  ran  sequentially 
without  interruption.  If  an  evaluator  had  a  distraction  during  a  Case  sequence,  or  needed  to 
reconsider  an  entry,  the  net  result  would  likely  have  been  a  missed  question. 
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Table  6.  System  Functionality  Question  #2. 1  (2-Plot  Graphic). 


2.1  Identifying  output  display  features. 

2-PIot  Graphic 

Case  I: 

No  Plume 

Case  II: 

Plume 

Case  III: 

No  Plume/Plume 

(a)  Two  area  maps 

50%  (5)* 

90%  (9)+ 

90%  (9) 

(b)  Time  stamp  of  wind  data. 

90%  (9)* 

90%  (9)+ 

100%  (10) 

(c)  The  height  of  output  winds  (Left  plot  header). 

90%  (9)* 

90%  (9)+ 

100%  (10) 

(d)  The  wind  model  streamlines  (Left  plot). 

90%  (9)* 

90%  (9)+ 

100%  (10) 

(e)  The  color-gradient  wind  velocities  (Left  plot). 

90%  (9)* 

90%  (9 )+ 

100%  (10) 

(f)  A  “No  Plume”  notice  (Right  plot).  [If  not  relevant,  put  N/A.] 

60%  (6)* 

N/A 

60%  (6) 

(g)  A  plume  overlay  (Right  plot) 

N/A 

90%  (9)+ 

80%  (8) 

(h)  A  plume  gradient  description  (Right  plot). 

60%  (6)* 

90%  (9)+ 

80%  (8) 

Comments:  2.1  L-REAC™  System  Output  {2-plot graphic) 

*Case  I:  Total  Evaluators  (a-h)  ->  #24  did  not  answer  this  question. 

+Case  II:  Total  Evaluators  (a-h)  ->  #16  penned  check  marks  were  difficult  to  read  for  this  question. 


3.3.3  Key  L-REAC™  System  Features  for  Decision  Makers 

Three  key  elements  of  the  L-REAC™  System  wind  and  plume  graphical  outputs  were  flagged 
for  assessment  in  the  functionality  questionnaire.  For  the  wind  field  graphics,  these  elements 
consisted  of: 

•  Finding  compass  North 

•  Determining  airflow  direction 

•  Determining  airflow  speed 

Compass  North  was  a  “key”  element  because  the  correct  map  orientation  is  critical  to  all 
subsequent  decision  aid  applications.  Identifying  the  current  wind  direction  and  speed  were  also 
flagged  as  crucial,  since  one  of  the  very  first  questions  normally  demanded  of  any  responder, 
following  an  airborne  toxin  release,  involves  the  current  winds.  Thus,  the  L-REAC™  user  will 
need  to  instantly  locate  and  retrieve  this  information. 

For  the  plume  graphical  output,  the  three  key  elements  were: 

•  Finding  compass  North 

•  Determining  the  (general)  hazardous  areas  generated  by  the  plume 

•  Determining  the  most/least  hazardous  areas 

Again,  map  orientation  is  crucial  to  a  successful  communication  of  decision  aid  results  and 
receiving  field  observations.  Interpreting  the  EUD  for  the  hazardous  areas,  as  well  as, 
identifying  the  most  and  least  hazardous  areas  is  the  primary  objective  for  this  decision  aid. 

Note:  When  there  is  not  enough  hazardous  materials  information  available  to  calculate  a  plume 
overlay,  the  perpetual  wind  field  still  empowers  incident  responders/commanders  with  decision- 
able  material. 
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In  the  next  two  sections,  the  evaluators  demonstrate  their  understanding  of  these  critical 
elements. 


3.3.3. 1  Wind  Field  Model  Output 

For  the  1 -Plot/Wind  Field  output  (see  table  7),  nearly  all  evaluators  were  able  to  find  compass 
North  for  each  of  the  three  Cases.  All  evaluators  defined  the  determining  of  the  airflow  direction 
as  “easy”  for  Cases  I  and  III.  In  Case  II,  one  person  selected  “sort  of”  easy,  while  all  others 
found  the  task  “easy”.  The  airflow  speed  was  a  little  more  challenging,  as  they  had  to  refer  to 
the  mesonet  (SAMS)  data  table  and  locate  the  measured  vector  on  their  visual  output.  Still,  the 
majority  labeled  the  task  as  “easy”.  Table  7  summarizes  the  results  from  the  1-Plot  evaluation  of 
the  Wind  Field  Model  output;  evaluator  comments  are  documented  after  the  table. 


Table  7.  System  Functionality  Question  #2.2.1  (1-Plot  Output). 


2.2.1  Evaluating  wind  field  model  output. 
1-plot  (Google  Earth)  Output 

Easy 

Sort  of 
Easy 

Difficult 

Not 

Relevant 

Total 

Number  of 
Evaluators 

(a)  Finding  compass  North  -  Case  I 

90%  (9) 

10%  (1)* 

0%  (0) 

0%  (0) 

10 

(a)  Finding  compass  North  -  Case  II 

90%  (9) 

10%  (1)+ 

0%  (0) 

0%  (0) 

10 

(a)  Finding  compass  North  -  Case  III 

90%  (9) 

10%  (1)A 

0%  (0) 

0%  (0) 

10 

(b)  Determining  airflow  direction  -  Case  I 

100%  (10) 

0%  (0) 

0%  (0) 

0%  (0) 

10 

(b)  Determining  airflow  direction  -  Case  II 

90%  (9) 

10%  (1)+ 

0%  (0) 

10%  (1)+ 

10+ 

(b)  Determining  airflow  direction  -  Case  III 

100%  (10) 

0%  (0) 

0%  (0) 

0%  (0) 

10 

(c)  Determining  airflow  speed  -  Case  I 

80%  (8) 

10%  (1)* 

10%  (1)* 

0%  (0) 

10 

(c)  Determining  airflow  speed  -  Case  II 

70%  (7) 

10%  (1)+ 

20%  (2)++ 

0%  (0) 

10 

(c)  Determining  airflow  speed  -  Case  III 

90%  (8) 

10%  (1) 

0%  (0) 

0%  (0) 

10 

Comments:  2.2.1  Wind  Field  Model  Output  ( 1-plot  graphic) 

*Case  I:  (a)  Sort  of  Easy  -  Blended  in  with  text  box.  Suggest  relocate  to  right  of  plot  map.  (#18) 

*Case  I:  (c)  Sort  of  Easy  -  /  know  that  north  is  always  up.  (#10) 

*Case  I:  (c)  Was  difficult  -  No  reference.  (#18) 

+Case  II:  (a)  Sort  of  Easy  -  Location  of  indicator.  (#18) 

+Case  II:  (b)  The  10  entries  to  this  question  come  from  #14  answering  “Was  Easy”  and  “Not  Relevant”. 

+Case  II:  (b)  Sort  of  Easy  -  Blue  arrows  were  very  small,  had  to  use  yellow  stream  arrows  to  tell  direction.  (#10) 

+Case  II:  (c)  Sort  of  Easy  -  By  looking  at  the  station  and  the  speed  there.  (#10) 

+Case  II:  (c)  Was  difficult  -  No  scale.  (#18) 

+Case  II:  (c)  Was  difficult  -  The  colors  indicating  the  concentrations  blend  into  the  background.  The  lines  are  extremely  thin. 

(Plume  concentrations)  (#24) 

ACase  III:  (a)  Sort  of  Easy  -  Same.  (#18) 

ACase  III:  (c)  Sort  of  Easy  -  No  scale.  (#18) 


For  the  2-Plot/Wind  Field  output  (see  table  8),  even  though  the  percentages  were  lower,  still  the 
majority  of  the  evaluators  found  compass  North  “easy”  for  each  of  the  three  Cases.  The  airflow 
direction  was  also  considered  “easy”  for  those  who  answered  the  question  (#24  did  not  answer 
the  question).  The  wind  speed  assessment  was  “easy”  for  the  majority  of  responses.  The 
consistent  increase  in  percentages  between  Case  I  and  II  implied  an  improved  understanding,  and 
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served  as  a  reminder  that  most  of  the  evaluators  were  still  assimilating  the  L-REAC™  Training  I 
information.  Table  8  displays  the  evaluator  results  for  the  System  Functionality  question  2.2.2. 


Table  8.  System  Functionality  Question  #2.2.2  (2-Plot  Output). 


2.2.2  Evaluating  wind  field  model  output. 
2-plot  (Wind  and  Plume)  Output 

Easy 

Sort  of  Easy 

Difficult 

Not 

Relevant 

Total 

Number  of 
Evaluators 

(a)  Finding  compass  North  -  Case  I 

70%  (7) 

10%  (1)* 

10%  (1)* 

0%  (0) 

9* 

(a)  Finding  compass  North  -  Case  II 

80%  (8) 

10%  (1)+ 

10%  (1)+ 

0%  (0) 

10 

(a)  Finding  compass  North  -  Case  III 

80%  (8) 

10%  (1)A 

10%  (1)A 

0%  (0) 

10 

(b)  Determining  airflow  direction  -  Case  I 

90%  (9) 

0%  (0) 

0%  (0) 

0%  (0) 

9* 

(b)  Determining  airflow  direction  -  Case  II 

100%  (10) 

0%  (0) 

0%  (0) 

0%  (0) 

10 

(b)  Determining  airflow  direction  -  Case  III 

100%  (10) 

0%  (0) 

0%  (0) 

0%  (0) 

10 

(c)  Determining  airflow  speed  -  Case  I 

70%  (7) 

20%  (2)** 

0%  (0) 

0%  (0) 

9* 

(c)  Determining  airflow  speed  -  Case  II 

90%  (8) 

10%  (1)+ 

0%  (0) 

0%  (0) 

10 

(c)  Determining  airflow  speed  -  Case  III 

90%  (9) 

10%  (1)A 

0%  (0) 

0%  (0) 

10 

Comments:  2.2.2  Wind  Field  Model  Output  (2-plot  graphic ) 

*Case  I:  Total  Evaluators  (a,  b,  c)  ->  #24  did  not  answer  this  question. 

*Case  I:  (a)  Sort  of  Easy  -  I  know  that  north  is  always  up.  (#10) 

*Case  I:  (a)  Was  difficult  -  No  “N"  indicator.  (#18) 

*Case  I:  (c)  Sort  of  Easy  -  Had  to  mark  a  known  speed  off  the  right  plot  then  measure  on  the  left  plot.  (#10) 

*Case  I:  (c)  Sort  of  Easy  -  Flipping  back  to  measure  the  red  arrow  to  get  a  speed.  (#15) 

+Case  II:  (a)  Sort  of  Easy  -  Knowing  that  North  is  always  up.  (#10) 

+Case  II:  (a)  Was  difficult  -  No  indicator.  (#18) 

+Case  II:  (c)  Sort  of  Easy  -  By  going  off  the  station  and  speed  there.  (#10) 

ACase  III:  (a)  Sort  of  Easy  -  Known  North.  (#10) 

ACase  III:  (a)  Was  difficult  —No  indicator.  (#18) 

ACase  III:  (c)  Sort  of  Easy  -  Using  color  chart  at  bottom.  (#10) 

33.3.2  Plume  Model  Output 

Case  I  did  not  use  a  plume  and  therefore  was  not  included  in  the  questionnaire’s  plume 
assessment  portion.  When  the  1-Plot  showed  a  plume  (Cases  II  and  III),  the  baseline 
understanding  answers  reported  that  evaluators  were  able  to  identify  the  plume  and  the  hazard- 
gradient  information.  To  better  focus  on  the  plume  output  details,  the  questionnaire  referenced 
only  the  2-Plot’s  use  of  the  plume  display. 

For  the  Plume/2-Plot  output  (see  table  9),  the  majority  of  evaluators  found  compass  North  “easy” 
(in  both  Cases  II  and  III).  Where  an  evaluator  reported  the  task  as  “difficult”,  this  answer  was 
understandable.  The  1-Plot  openly  shows  “North”  by  using  a  labeled  arrow  pointing  up  (to  the 
top  of  the  page).  To  reduce  visual  clutter,  the  2-Plot  also  uses  “up”  as  North  but  this  orientation 
is  implied  from  the  1-Plot  and  taught  as  a  part  of  the  training  class. 
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The  hazardous  areas  were  “easy”  to  locate  for  the  majority  of  evaluators.  A  user  request  for 
measured  distance  was  raised  in  this  section.  The  L-REAC™  System  was  designed  to  use 
landmarks  from  the  overhead  satellite  imagery.  There  are  horizontal  scales  on  the  2-Plot/wind 
field.  The  2-Plot/Plume  output  comes  with  a  subtle  white  distance  scale  on  map,  though  this 
feature  is  not  always  discernable  at  the  remote  user’s  terminal.  The  scale  is  available  on  the 
operator’s  tenninal  display. 

The  most  and  least  hazardous  areas  were  considered  “easy”  to  find  by  the  majority  of  evaluators. 
The  suggestion  to  darken  the  contrast  within  the  plume  gradients  was  addressed  in  the 
“immediate  improvements”  section.  Table  9  tabulates  the  results  from  the  plume  interpretation 
question. 


Table  9.  System  Functionality  Question  #2.3.2  (2-Plot  Output). 


2.3.2  Evaluating  Wind  Field  Model  Output. 
2-plot  (Wind  and  Plume)  Output 

Easy 

Sort  of 
Easy 

Difficult 

Not 

Relevant 

Total 

Number  of 
Evaluators 

(a)  Finding  compass  North  -  Case  II 

80%  (8) 

0%  (0) 

10%  (1)+ 

0%  (0) 

9+ 

(a)  Finding  compass  North  -  Case  III 

60%  (6) 

0%  (0) 

10%  (1)A 

0%  (0) 

7A 

(b)  Determining  hazardous  areas  generated  by 
plume  -  Case  II 

80%  (8) 

0%  (0) 

10%  (1)+ 

0%  (0) 

9+ 

(b)  Determining  hazardous  areas  generated  by 
plume  -  Case  III 

70%  (7) 

10%  (1)A 

0%  (0) 

0%  (0) 

8A 

(c)  Determining  most/least  hazardous  areas  - 
Case  II 

80%  (8) 

10%  (1)+ 

0%  (0) 

0%  (0) 

9+ 

(c)  Determining  most/least  hazardous  areas  - 
Case  III 

70%  (7) 

10%  (1)A 

0%  (0) 

0%  (0) 

8A 

‘Comments:  2.3.2  Evaluating  the  Plume  Model  (2-plot  graphic) 

*Case  I  did  not  use  the  plume  model  and  is  therefore  not  listed  in  this  table. 

+Case  II:  Total  Evaluators  (a,  b,  c)  ->  #8  did  not  answer  this  question. 

+Case  II:  (a)  Was  difficult  -  No  indicator.  (#18) 

+Case  H:  (b)  Was  difficult  -  No  distance  scale  to  reference.  (#18) 

+Case  II:  (c)  Sort  of  Easy  -  Lines  very  thin,  better  when  you  filled  in  the  red  area.  (#1 8) 

ACase  III:  Total  Evaluators  (a,  b,  c)  ->  #13  did  not  answer  this  question. 

ACase  III:  Total  Evaluators  (a)  ->  #16  did  not  answer  the  Compass  North  (a)  part  of  this  question. 

ACase  III:  (a)  Was  difficult  -  No  indicator.  (#18) 

ACase  III:  (b)  Sort  of  Easy  -  Would  help  to  fill  most  hazardous  with  color.  (#18) 

ACase  III:  (c)  Sort  of  Easy  -  Should  fill  area  with  AEGL-3  [Acute  Exposure  Guideline  Level  -  3]  with  red.  (#18) 

3.4  Future  Vision  and  Final  Comments  from  the  Evaluators 

The  Administrative  Questionnaire  concluded  with  an  invitation  to  suggest  improvements  for  the 
current  L-REAC™  System,  and  to  make  any  “Final  Comments”.  The  following  improvements 
were  recommended  by  the  evaluators: 

•  Phone  applications,  web  access  if  evacuated  off  post.  (#3) 
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•  Add  N-hour  and  the  type  of  chemicals  released.  (#8) 

•  The  ability  to  cover  a  wider  area  &  zoom  in  to  your  area  of  interest  like  on  Google  Earth. 
Off  main  cantonment  areas  like  the  F.B.R.  (Fast  Burst  Reactor)  for  use  during  a 
radiological  plume.  (#10) 

•  More  locations  providing  real-time  data  (red  arrow);  APP  creation  for  smart  phones. 

(#11) 

•  My  future  vision  for  L-REAC™  would  be  a  system  with  real  time  display  to  incorporate 
multiple  Plume  data.  (#13) 

•  Would  like  to  see  graduation  to  hand-held  that  soldier  could  use  in  the  field.  (#16) 

•  (1)  Ability  to  search  for  multiple  chemical  presence;  (2)  Continuous/autonomous  system 
monitoring/operational;  (3)  Monitor  multiple  locations.  (#18) 

The  Evaluator’s  final  comments  included: 

•  Good  &  easy  system  to  use.  Is  this  the  system  used  by  the  weather  people  on  TV  that  show 
wind  maps  of  the  area;  or  can  they  be  tied  in  together.  (#10) 

•  Keep  the  visuals  (GUI)  [Graphical  Users  Interface]  clean.  (#11) 

•  Keep  up  the  great  work.  (#14) 

•  I  think  the  IOC  could  use  a  bigger  map  for  the  wall,  so  that  when  things  get  hopping,  we 
don ’t  have  a  crowd  behind  a  computer.  Also,  it  would  help  (to)  show  the  bigger  picture. 
(#15) 

•  Great  demonstration.  (#16) 

•  Incredible  what  comes  out  of  burning  crap!  Also,  would  like  one  day  a  week  for  a  month 
or  so  to  train  operators  by  having  L-REAC™  on-line  and  processing  data  requests  from 
the  IOC.  (#18) 


4.  Discussion 


The  purpose  for  this  L-REAC™  System  improvement  evaluation  task  was  to  solicit  practical  and 
visionary  improvements  from  volunteer  and  professional  emergency  first  responders.  The 
evaluation’s  implementation  also  served  as  an  opportunity  for  the  Training  I  students  to  test  their 
newly  acquired  skills  with  the  L-REAC™  System  output.  Keeping  these  two  perspectives  in 
mind,  the  results  were  most  encouraging. 


26 


4.1  Detailed  Review  of  the  L-REAC™  System  Product 

Beginning  with  the  detailed  questionnaire,  the  communication  method  for  gaining  access  to  the 
L-REAC™  System  output  was  easily  understood  and  implemented.  The  basic  features  of  the 
1-Plot  and  2-Plot  EUD  outputs  were  easily  located.  Key  features  from  which  decisional  actions 
are  anticipated  were  defined  as  “easy”  to  find,  by  the  majority  of  evaluators. 

For  the  wind  field  output,  an  isolated  “difficulty”  in  finding  the  wind  speed  in  the  1 -Plot/Case  I 
and  II  scenarios  was  linked  to  a  zoomed-in,  narrow  field-of-view  image  with  no  available 
reference  vectors.  L-REAC™  has  no  control  over  the  number  or  placement  of  mesonet  reference 
vectors.  Another  isolated  “difficulty”  flagged  the  locating  of  compass  North  on  a  2-Plot  output. 
This  response  was  assessed  to  be  a  function  of  student  learning.  As  explained  earlier,  1-Plot 
images  have  a  labeled  arrow  pointing  up  (top  of  page)  indicating  North.  To  reduce  visual  clutter, 
the  simultaneously  issued  2-Plot  image  implies  North  as  up  (top  of  page).  This  presumed  pattern 
for  the  2-Plot  was  explained  during  Training  I.  Aside:  some  Training  I  students  took  the 
training  after  completing  a  10-hr  shift,  which  no  doubt  challenged  the  ability  to  absorb  all  the 
many  details  presented  during  the  training  session. 

For  the  plume  output  evaluation,  the  majority  of  evaluators  found  the  key  features  of  the  2-Plot 
easily.  The  “difficulty”  reported  concerned  color  discernment  and  a  North  reference.  The  latter 
was  explained  above.  The  color  discernment  was  resolved  as  part  of  the  implemented  evaluation 
results  prompted  after  the  201 1  Abrams  Fire  Incident.  That  is,  tailored  color  hues  and  outlines 
more  conducive  to  a  desert  background  image  and  the  viewing  of  a  computer  screen  under 
natural  sunlight,  were  selected  for  the  EUD  output. 

4.2  Overall  System  Evaluation 

Evaluators  unanimously  agreed  that  they  would  recommend  the  L-REAC™  System  to  be 
purchased.  Their  reasons  were  listed  in  section  3.2.1.  Two  volunteered  responses  in  particular 
capture  the  primary  system’s  goal  and  greatest  challenge.  These  were: 

•  It  (the  L-REAC™  System)  can  save  lives! 

•  This  tool’s  ability  to  determine  wind  flow  around  structures  is  very  useful  for  Fire  Fighters 
and  EM  (emergency  management).  Selling  the  product  would  be  great;  too  bad  those  that 
need  it  most  would  most  likely  not  afford  it. 

The  evaluator  applications  for  the  L-REAC™  System  consisted  of  CBRNE  Force  Protection 
applications,  IOC,  EOC,  Fire/Rescue  and  Emergency  Dispatch.  All  evaluators  reported  that  the 
L-REAC™  System  would  be  an  asset  to  their  profession/application. 

Strengths  cited  for  the  System  included  five  comments  which  praised  the  ease  of  use, 
understandability  of  end  products,  and  the  “great”  potential  of  the  System.  The  (near)  real-time 
capability  was  favorably  emphasized  in  two  evaluator  comments,  as  well  as,  the  applicability  for 
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real  world  usages.  System  strengths  applicable  to  the  evaluator’s  profession  included  both  the 
wind  field  and  plume  elements.  The  “ease  of  use”  was  again  emphasized  as  a  significant 
strength  by  these  first  responders. 

To  better  inform  the  System  designers  regarding  the  operational  emergency  management 
environment,  two  scenarios  for  integrating  the  L-REAC™  System  into  the  emergency  response 
community  were  offered: 

•  Multiple  L-REAC™  Systems  in  various  organizations. 

•  A  single  L-REAC™  System  with  multiple  users  (representing  different  organizations). 
The  evaluators  offered  a  third  arrangement: 

•  Two  (or  three)  L-REAC™  Systems:  One  organization  would  serve  as  the  primary 
L-REAC™  System  operator  with  multiple  users  (representing  different  organizations). 
The  second  system  would  be  installed  at  a  different  location,  and  would  serve  as  a  backup 
unit,  should  the  primary  unit  suddenly  loose  power  or  network  connectivity. 

The  most  favored  arrangement  was  to  have  multiple  organizations  own  an  L-REAC™  System. 
Based  on  current  economic  restrictions,  the  author  suspects  that  the  ideal  arrangement  for 
integrating  an  L-REAC™  System  into  the  emergency  response  community  would  be  the  third 
suggested  option. 

Under  a  multiple  organization  ownership  design,  the  following  agencies  were  unanimously 
suggested  for  owning  a  System:  the  Fire  Dept/Medical  and  the  IOC.  Seventy-eight  percent  of 
the  evaluators  included  the  EOC.  The  least  recommended  owner  was  installation  commanders. 

With  a  single  unit  ownership  design,  the  unanimous,  top  recommendation  for  receiving  system 
output  was  the  Fire  Dept/Medical.  The  IOC,  EOC,  Police,  IC  on  location,  IC  in  IOC  and 
Garrison  Commander  were  also  suggested  by  the  majority  of  evaluators.  The  least 
recommended  for  receiving  user  output  was  the  installation  Commanding  General. 

One  of  the  evaluator  suggestions  for  potential  owners  and  users  of  the  L-REAC™  System  was 
the  “Dispatch”  office.  Dispatch  has  a  distinct  function  from  the  IOC  and  EOC,  so  this  office  is 
definitely  a  suggestion  that  needs  to  be  considered. 

Directions  for  “how  to  access  the  L-REAC™  System  remotely”  were  sent  to  each  evaluator. 
These  instructions  were  unanimously  evaluated  as  “easy  to  follow”  and  the  System  output  was 
labeled  as  timely.  The  output  details  were  assessed  as  “enough”  (versus  “too  many  details”). 
The  Operator’s  zoom-in  capability  was  considered  very  useful  in  the  emergency  mgt 
(management)  arena. 

Focusing  on  the  System  output  details,  almost  all  evaluators  considered  the  listed  1-Plot  and 
2-Plot  features  as  useful.  Seventy-eight  percent  indicated  ALL  information  as  “most  useful.” 
The  lowest  rated  feature  (89%)  was  the  “No  Plume”  notice. 
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4.3  Evaluator  Future  Vision  and  “Final  Comments” 

The  evaluators  provided  their  future  L-REAC™  vision  by  suggesting  System  improvements. 
Several  items  suggested  reinforced  designs  already  flagged  for  future  L-REAC™  Systems.  For 
example:  Regarding  the  phone  applications  suggestions — In  2010-2011,  a  feasibility  test  was 
successfully  conducted,  which  overlaid  the  L-REAC™  System  plume  output  on  an  iPod’s  AOI 
display.  Subsequently,  plans  for  sending  the  output  to  an  Android  (an  Army  endorsed  hand-held 
phone)  are  being  considered.  The  ultimate  goal  of  this  effort  is  to  bring  the  hand-held 
technology  to  our  Soldiers  in  the  field. 

Regarding  the  request  to  cover  a  wider  AOI — The  L-REAC™  System  now  includes  the  ability  to 
cover  three  scales  of  wind  field  AOIs.  The  outer  extremes  of  these  scales  include  the  building 
scale  (covering  ~2-km  width)  and  the  regional  scale  (covering  ~50-km  width).  See  section  3.1. 
The  latter  scale  covers  the  suggested  FBR  site  mentioned  in  the  evaluator  comments. 

The  ability  to  provide  more  measured  data  (red  arrows)  is  a  function  of  the  organizations  that 
sponsors  the  mesonet  resources.  While  L-REAC™  has  no  direct  influence  on  those  decisions, 
should  the  opportunity  to  express  the  evaluator’s  suggestion  arise,  it  will  be  shared. 

Searching  for  multiple  chemicals,  such  as  using  a  chemical  sniffer  sensor,  was  considered  in  the 
original  L-REAC™  System  design  for  the  Sensor  Module.  Due  to  budgetary  constraints,  this 
option  had  to  be  tabled.  Currently,  integrating  sniffer  technology  is  still  pending  funding. 

Being  able  to  display  multiple  plumes  is  another  vision  that  was  in  the  original  L-REAC™ 
System  design.  The  implementation  of  this  vision  began  with  the  inclusion  of  measured 
quantities  in  the  output  display.  The  designers  purposefully  did  not  impose  a  spatial-relevance 
limitation  on  the  mesonet  data  listing  on  the  output.  This  decision  was  to  accommodate  the 
potential  for  multiple  fire/toxic  release  locations  occurring  simultaneously. 

Continuing  with  the  concept  of  multiple  plumes,  the  original  L-REAC™  System  design  included 
running  multiple  plume  (and  wind)  models,  concurrently.  The  core  L-REAC™  quick  response 
models  would  continue  to  run  as  designed,  while  a  second  tier  of  plume  models  would  be 
executed  as  background  computer  jobs.  Within  these  slower,  second  tiered  plume  models, 
multiple  chemical  releases  could  be  addressed  without  impacting  the  immediate  decisions  still 
needing  to  be  made  from  the  core  modules/models. 

The  evaluator  “Final  Comments”  were  most  encouraging  to  the  L-REAC™  System  designers. 
The  suggested  bigger  map  for  the  IOC  wall  has  been  addressed  in  that  the  referenced-IOC  can 
now  project  the  L-REAC™  Output  onto  their  wall  via  their  local  computers/IOC  wall  projector. 

The  “burning  chemicals”  reference  in  the  last  comment  stems  from  a  2009  CNN  article,  which 
reported  that  the  Department  of  Defense  (DoD)  found  bum  pits,  which  were  used  instead  of 
incinerators  on  some  bases/outposts  in  Iraq  and  Afghanistan  in  2007,  could  have  caused  short 
term  effects  to  the  U.S.  Soldiers.  The  publishing  of  this  article  coincided  with  the  initial  design 
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and  development  of  the  L-REAC™  System  “Proof  of  Concept”.  The  article  reinforced  the 
relevancy  of  this  decision  aid  to  the  U.S.  Army  soldier  and,  to  the  trainee’s  credit,  was  included 
as  one  of  the  earliest  slides  in  the  Training  I  briefing.  The  website  article  is: 
http://www.cnn.com/2009/PQLITICS/12/18/military.bum.pits/index.htm  (CNN,  2009). 

In  response  to  the  evaluator’s  training  request,  the  L-REAC™  Group  has  since  offered  training 
to  all  Evaluation-participating  organizations.  The  sessions  were  initially  conducted  over  a  one- 
month  time  period  in  fiscal  year  2011  (FY11).  Five  levels  of  training  were  offered:  Levels  1-2 
prepared  the  student  to  be  an  L-REAC™  System  user  (able  to  correctly  interpret  the  L-REAC™ 
output).  Levels  3-5  prepared  the  trainee  for  being  an  L-REAC™  System  operator.  In  fiscal  year 
2012  (FY12),  additional  requests  for  training  were  received  and  completed.  At  the  time  of  this 
report,  there  have  been  41  participating  trainees  for  FY 11-12,  and  nine  trainees  have  reached  the 
Operator  responsibility. 


5.  Summary 


ARL  has  been  studying  urban  environments  in  support  of  the  increased  urban  warfare.  Within 
the  first  decade  of  2000,  three  progressively  more  complex  urban  field  studies  were  conducted  at 
WSMR,  NM.  These  studies  examined  airflow  and  stability  around  urban  buildings.  Concurrent 
with  the  final  study  WOWS ,  emergency  first  response  drills  were  executed.  During  WOWS ,  the 
need  for  first  responders  to  have  access  to  near  real-time  atmospheric  data  was  noted.  This 
observation  was  later  documented  in  ARL-TR-5706  (Vaucher,  201 1),  which  also  justified  the 
current  ARL-WSMR  safety  procedure  update  and  the  use  of  an  active  L-REAC™  System  by 
ARL. 

ARL  responded  to  the  civilian/military  emergency  responder  need,  by  designing  the  L-REAC™ 
System.  From  2009  to  201 1,  the  viewgraph  concept  was  converted  into  a  “Proof  of  Concept,” 
Prototype  and  finally,  an  Operational  System.  The  current  L-REAC™  System  is  an  automated, 
24/7  emergency  response  decision  aid  designed  for  airborne  toxic  release  incidents.  The  basic 
design  of  the  L-REAC™  System  includes  five  modules:  the  Sensor,  QC,  Model,  EUD  and 
Archive  Modules.  The  Sensor  module  continuously  acquires  meteorological  data  used  as  input 
by  both  a  wind  field  and  plume  model.  These  models  continually  run  and  produce  a  1-Plot  and 
2-Plot  html  output,  which  is  accessible  by  authorized  end  users.  The  output  shows  an  overview 
map  of  an  AOI,  with  wind  field  and  plume  (when  applicable)  overlays.  These  overlays  are 
updated  every  2-8  min,  depending  on  the  resolution  of  the  wind  field  model.  An  archive 
continually  saves  both  the  measured  data  and  (when  the  user  has  selected  this  option)  the  image 
output.  The  System  is  designed  to  run  attended  by  an  operator  or  without  an  operator. 

As  part  of  the  technology  transfer  process,  the  L-REAC™  System  was  subjected  to  several 
informal  and  formal  evaluations.  The  first  evaluation  was  an  informal  demonstration  of  the 
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System  to  about  30  GEMS  high  school  students/teachers,  who  used  the  system  to  assess  the  wind 
flow  and  safe  zone  locations.  Several  Force  Protection  Exercises  and  Drills  have  provided  user 
feedback  throughout  the  development  phases.  In  April  2011,  the  L-REAC™  System  participated 
in  the  “real-world”  event  called,  “the  Abrams  Fire.”  Training  sessions  were  subsequently 
requested  by  the  emergency  first  response  organizations.  Within  these  training  sessions,  the  first 
formal  L-REAC™  System  improvement  evaluations  were  conducted.  The  results  are  the  subject 
of  this  technical  report. 

The  evaluations  were  organized  into  two  questionnaires:  An  Administrative  Questionnaire, 
which  looked  at  the  System  from  an  overview  and  future  vision  perspective.  The  second 
questionnaire  focused  the  System  Functionality,  and  included  a  more  detailed  review  of  the 
System  output.  This  questionnaire  also  served  as  an  informal  test  for  the  students  who  had  just 
completed  their  Training  I  course.  The  Functionality  Questionnaire  was  divided  into  three 
cases/scenarios:  Case  I  (No  Plume  Scenarios),  Case  II  (Plume  Scenarios)  and  Case  III  (No 
Plume/Plume  Scenarios). 

The  results  of  the  Administrative  Questionnaire  reported  that,  from  the  remote  communications 
through  the  continuously  updated  wind  and  plume  images,  the  system  was  easily  understood  and 
implemented.  The  few  isolated  items  flagged  as  “difficult”  could  be  explained  in  the  context  of 
students  learning  (this  evaluation  was  the  first  time  some  had  seen  the  System  in  operation).  The 
viewing  output  difficulty  was  addressed  when  the  color  contrasts  were  enhanced  for  desert 
environment  backgrounds. 

Evaluators  unanimously  stated  that  they  would  recommend  the  system.  The  explanations  for 
why  they  would  recommend  the  System  captured  the  System  objective  of  “saving  lives,”  and  the 
challenge  of  finding  funding  resources  to  support  system  development  costs. 

The  evaluator  applications  for  the  L-REAC™  System  consisted  of  CBRNE  Force  Protection 
applications,  IOC,  EOC,  Fire/Rescue  and  Emergency  Dispatch.  All  evaluators  reported  that  the 
L-REAC™  System  would  be  an  asset  to  their  profession/application. 

System  strengths  cited  included  five  comments  which  praised  the  ease  of  use,  understandability 
of  end  products,  and  the  “great”  potential  of  the  System. 

To  edify  the  System  designers  regarding  how  the  L-REAC™  might  be  integrated  into  the 
operational  emergency  environment,  two  scenarios  were  offered: 

•  Multiple  L-REAC™  Systems  in  various  organizations. 

•  A  single  L-REAC™  System  operated,  with  multiple  users  (representing  different 
organizations). 
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The  evaluators  offered  a  third  arrangement: 

•  Two  (or  three)  L-REAC™  Systems:  One  organization  would  serve  as  the  primary  operator 
with  multiple  users  (representing  different  organizations).  The  second  system  would  be 
installed  at  a  different  location,  and  would  serve  as  a  backup  unit,  should  the  primary  unit 
suddenly  loose  power  or  network  connectivity. 

While  the  most  favored  arrangement  was  to  have  multiple  organizations  own  an  L-REAC™ 
System,  economics  may  limit  the  arrangement  to  the  third  option.  With  multiple  organizations 
each  owning  a  System,  the  Fire  Dept/Medical  and  the  IOC  were  unanimously  selected  as 
potential  owners.  Over  three-fourths  of  the  evaluators  included  the  EOC,  as  a  potential  owner. 
The  least  recommended  prospective  owners  were  the  installation  command  offices. 

With  a  single,  central-owner  design,  the  unanimous  top  recommendation  for  a  user  was  the  Fire 
Dept/Medical.  The  IOC,  EOC,  Police,  IC  on  location,  IC  in  IOC  and  Garrison  Commander  were 
also  suggested  for  users  by  the  majority  of  evaluators.  The  least  recommended  user  was  the 
installation  Commanding  General.  The  “Dispatch”  office  was  suggested  as  both  an  owner  and 
user. 

“How  to  access  the  L-REAC™  System  remotely”  directions  were  unanimously  evaluated  as 
“easy  to  follow”  and  the  System  output  was  flagged  as  timely.  The  output  details  were  assessed 
as  “enough”  (versus  “too  many  details”).  The  Operator’s  zoom  in  capability  was  considered 
very  useful  in  the  emergency  mgt  (management)  arena. 

Focusing  on  the  output  details,  the  majority  of  evaluators  considered  the  listed  1-Plot  and  2-Plot 
features  as  useful.  Seventy-eight  percent  indicated  ALL  information  as  most  useful.  The 
“lowest”  rated  feature  (selected  as  useful  by  89%  of  the  evaluators)  was  the  “No  Plume”  notice. 


6.  Final  Comments  by  Author  and  Evaluators 


All  the  evaluator  comments  have  been  extremely  helpful  is  calibrating  the  value  of  the  L- 
REAC™  System  to  the  emergency  response  community.  While  funding  may  limit  its  future 
development,  the  fact  that  the  System  has  already  proven  to  have  a  positive  impact  on  both  the 
local  ARL  safety  procedures  and  surrounding  military/civilian  communities  indicate  that,  at  the 
right  time  (when  funded),  this  technology  will  continue  to  make  a  constructive  difference.  And, 
with  proper  support,  the  System  is  still  a  candidate  for  use  by  ARL’s  primary  customer,  the  U.S. 
Army  Soldiers. 
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In  the  meantime,  the  “final  comments”  for  this  report  are  provided  by  the  evaluators: 

•  (The  L-REAC™  System  is)  Good  &  easy  system  to  use.  Is  this  the  system  used  by  the 
weather  people  on  TV  that  show  wind  maps  of  the  area;  or  can  they  be  tied  in  together. 
(#10) 

•  Keep  the  visuals  (GUI)  clean.  (#11) 

•  Keep  up  the  great  work.  (#14) 

•  I  think  the  IOC  could  use  a  bigger  map  for  the  wall,  so  that  when  things  get  hopping,  we 
don ’t  have  a  crowd  behind  a  computer.  Also,  it  would  help  (to)  show  the  bigger  picture. 
(#15) 

•  Great  demonstration.  (#16) 

•  Incredible  what  comes  out  of  burning  crap!  Also,  would  like  one  day  a  week  for  a  month 
or  so  to  train  operators  by  having  L-REAC™  on-line  and  processing  data  requests  from 
the  IOC...  (#18) 
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Appendix  A.  System  Functionality  Questionnaire-Evaluator  Results 


The  System  Functionality  Questionnaire  allowed  the  voluntary  and  professional  emergency  first 
responders  to  provide  a  detailed  review  of  the  L-REAC™  System  products.  This  review  was 
gleaned  through  three  scenarios  or  Cases,  each  representing  foundational  functions  for  the  L- 
REAC™  System  output.  The  Cases  included: 

Case  I:  (No  Plume  Scenarios) 

•  STATUS:  Wind  Field  Model  and  No  Plume. 

•  SCENARIOS:  Entire  AOI,  3  Mesonet  Locations,  Medical  Facility,  a  User  Selection. 

Case  II:  (Plume  Scenarios) 

•  STATUS:  Wind  Field  Model  and  Pre-determined  Plume. 

•  SCENARIOS:  Training  site  -  large  image  (1)  and  zoomed-in  images  (3). 

Case  III:  (No  Plume/Plume  Scenarios,  with  multiple  locations) 

•  STATUS:  Wind  Field  Model  with  No  Plume  &  Plume  scenarios. 

•  SCENARIOS:  Training  site-No  plume  &  Plume;  Plume  at  user-selected  site  within  AOI. 

For  an  explanation  of  the  System  Functionality  Questionnaire  and  the  three  Cases,  see  section 

2.2.1. 

The  statistical  results  presented  in  the  L-REAC™  System  Evaluation:  Case  I,  II,  II  ( System 
Functionality)  include  data  and  comments  from  all  ten  System  Functionality  Questionnaires 
returned.  Where  an  evaluator  skipped  or  missed  a  question,  a  note  was  placed  after  the  question, 
but  the  tally  was  still  based  on  ten  total  submissions. 

Note:  If  the  tallies  had  been  based  on  the  actual  number  of  evaluators  responding  to  the 
question,  the  tallies  would  have  been  higher  in  magnitude. 
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L-REAC™  SYSTEM  EVALUATION:  CASE  I  -  System  Functionality 


Case  I:  The  1-Plot  and  2-Plot  Evaluation,  No  Plume:  (30  minutes) 

•  STATUS :  Wind  Field  Model  (5m  resolution)  and  No  Plume. 

•  SCENARIOS:  WSMRPost,  1622,  Museum,  Golf  Course,  Medical  Clinic,  User  Selection. 

Evaluator  #:  3,8,  10, 11, 13,  14,  15,  16,  18,  24  [  10  total  ] 

Evaluation  Date/Time:  _  _ MT 

1.  Access  to  L-REAC™  System  Output:  (#24:  Not  answer  1.1  and  1.2  questions.) 

1.1.  Describe  your  experience  with  the  DMZ  network  connection  (L-REAC™  output  access): 
90%  (9)  -  The  DMZ  connection  ran  without  any  interruptions. 

_  The  DMZ  Connection  ran  with  interruptions,  (please  describe) _ 


_ Other: _ 

1.2  Frequency  of  output  updates: 

70%  (1)  -  The  frequency  of  output  updates  was  sufficient. 

10:  for  this  case 

10%  (11  -  The  frequency  of  output  updates  needs  to  be  more  often. 

_ -  The  frequency  of  output  updates  can  be  less  often. 

10%  (1)  -  Other _ 

3:  Was  Sufficient,  temporarily  dropped  out. 

2.  The  L-REAC™  System  Output: 

2. 1  During  the  evaluation  period,  which  of  the  following  was  seen  on  your  output  display? 
(Select  all  that  apply): 

1- plot  graphic: 

100%  (10)  -  An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

100%  (101  -  The  date  and  time  stamp. 

100%  (10)  -  The  height  of  the  output  winds. 

100%  (10)  -  The  wind  field  model  wind  arrows  (blue). 

100%  (101  -  The  wind  field  model  streamlines  (yellow). 

90%  (9)  -  The  measured  wind  arrows  (red). 

90%  (91  -  A  “No  Plume”  notice.  [If  not  relevant,  put  N/A  on  line  ] 

90%  (91  -  The  WSMR  Surface  Automated  Meteorology  System  (SAMS)  data  list  (text  box). 

2- plot  graphic:  (#24:  Not  answer  2.1  2-plot  graphic.) 

n/a  50%  (5)  -  Two  area  maps. 

90%  (91  -  Time  stamp  of  wind  data. 

90%  (91  -  The  height  of  output  winds  (Left  plot  header). 

90%  (9)  -  The  wind  model  streamlines  (Left  plot). 

90%  (9)  -  The  color-gradient  wind  velocities  (Left  plot). 

n/a  60%  (61  -  A  “No  Plume”  notice  (Right  plot).  [If  not  relevant,  put  N/A  on  line.] 
n/a  60%  (61  -  A  plume  gradient  description  (Right  plot). 

1-1 


38 


2.2  Evaluating  the  Wind  Field  Model  output:  (Questions  2.2. 1  and  2.2.2) 

2.2. 1  On  the  1-plot  (Google  Earth)  output. . . 

1-Plot  -  Finding  compass  north:  (select  one) 

90%  (9)-Was  easy. 

10%  (l)-Was  sort  of  easy.  Why?  18:  Blended  in  w/text  box.  Suggest  relocate  to  right  of  plot  map. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1-Plot  -  Determining  the  direction  for  the  airflow:  (select  one) 

100%  (10)  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1- Plot  -  Determining  the  speed  of  the  airflow:  (select  one) 

80%  (8)-Was  easy. 

10%  (l)-Was  sort  of  easy.  Here’s  why  11:  Had  to  reference  previous  picture  with  red  arrow 
10%  (11- Was  difficult.  Here’s  why  18:  No  reference. 

_ Was  not  relevant.  Here’s  why _ 

2.2.2.  On  the  2-plot  (Wind  and  Plume)  output. . .  [24:  Not  answer  question.] 

2- Plot,  Wind  Field  Plot  (left  side)  -  Finding  compass  north:  (select  one) 

70%  (71-Was  easy. 

10%  (11- Was  sort  of  easy.  Here’s  why  10:  I  know  that  north  is  always  up 
10%  (l)-Was  difficult.  Here’s  why  18:  No  “N”  indicator 
_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  airflow  direction:  (select  one) 

90%  (9)-Was  easy. 

_  Was  sort  of  easy.  Here’s  why _ 

_ Was  difficult.  Here ’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  speed  of  the  airflow:  (select  one) 
70%  (71-Was  easy. 

20%  (2)- Was  sort  of  easy.  Here’s  why 

10:  Had  to  mark  a  known  speed  off  the  right  plot  then  measure  on  the  left  plot. 
15:  Flipping  back  to  measure  the  red  arrow  to  get  a  speed. 

_ Was  difficult.  Here ’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1-2 
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L-KKAC™  SYSTEM  EVALUATION:  C  ASK  II  System  Functionality 


Case  II:  The  1-Plot  and  2-Plot  Evaluation,  Plume:  (20  minutes) 

•  STATUS:  Wind  Field  Model  (5m  resolution)  and  Pre-determ ined  Plume. 

•  SCENARIO:  Bldg  1622  -  large  image  (1)  and  zoomed-in  images  (3). 

Evaluator  #:  3.8,10,11, 13,14. 15. 16,18,24  [  10  total  ] 

Evaluation  Date/Time:  _  _ MI 

I .  Access  to  L-REAC™  System  Output:  (#8  and  24:  Not  answer  1.1  and  1.2  questions.) 
1.1.  Describe  your  experience  with  the  DMZ  network  connection  (L-REAC™  output  access): 
70%  (7)-The  DMZ  connection  ran  without  any  interruptions. 

10%  (l)-Tlie  DMZ  Connection  ran  with  interruptions,  (please  describe) _ 


_ Other: _ 

1.2  Frequency  of  output  updates: 

60%  (6  )-The  frequency  of  output  updates  was  sufficient. 

10:  With  use  of  the  F-5  key  every  now  and  then. 

20%  (2)-Tlie  frequency  of  output  updates  needs  to  be  more  often. 

_ Tbe  frequency  of  output  updates  can  be  less  often. 

_ Other _ 

2.  The  L-REAC™  System  Output: 

2. 1  During  the  evaluation  period,  which  of  the  following  was  seen  on  your  output  display? 
(Select  all  that  apply):  (#16:  Difficult  to  read  ink  answers,  may  have  impacted  2-plot.) 

1- plot  graphic: 

100%  (10)  -  An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

100%  (10)  -  The  date  and  time  stamp. 

70%  (7)  -  The  height  of  the  output  winds.  10:  I  missed  the  red  arrow  on  the  first 
screen  did  not  see  it  on  the  second  zoomed  in  screen. 

90%  (9)  -  The  wind  field  model  wind  arrows  (blue). 

90%  (9)  -  The  wind  field  model  streamlines  (yellow). 

70%  (7)  -  The  measured  wind  arrows  (red). 

100%  (10)  -  A  plume  overlay. 

100%  (10)  -  A  plume  gradient  description  of  the  gradients  used. 

90%  (9)  -  The  WSMR  Surface  Automated  Meteorology  System  (sams)  data  list  (textbox). 

2 - plot  graphic: 

90%  (9)  -  Two  area  maps. 

90%  (9)  -  Time  stamp  of  wind  data. 

90%  (9)  -  The  height  of  output  winds  (Left  plot  header). 

90%  (9)  -  The  wind  model  streamlines  (Left  plot). 

90%  (9)  -  The  color-gradient  wind  velocities  (Left  plot). 

90%  (9)  -  A  plume  overlay  (Right  plot). 

90%  (9)  -  A  plume  gradient  description  (Right  plot). 

n-i 
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2.2  Evaluating  the  Wind  Field  Model  output:  (Questions  2.2. 1  and  2.2.2) 

2.2. 1  On  the  1-plot  (Google  Earth)  output. . . 

1-Plot  -  Finding  compass  north:  (select  one) 

90%  (91  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.  Here’s  why  18:  Location  of  indicator 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1-Plot  -  Determining  the  direction  for  the  airflow:  (select  one) 

90%  (9)  -  Was  easy. 

10%  (11  -  Was  sort  of  easy.  Here’s  why 

10:  Blue  arrows  were  very  small,  had  to  use  yellow  stream  arrows  to  tell  direction. 
_ Was  difficult.  Here’s  why 

10%  ( 1)  -  Was  not  relevant.  Here’s  why  NOTE,  #14:  “was  easy”  &  “not  relevant”. 

1- Plot  -  Determining  the  speed  of  the  airflow:  (select  one) 

70%  (7)  -  Was  easy. 

10%  (11  -  Was  sort  of  easy.  Here’s  why  10:  By  looking  at  the  station  &  the  speed  there. 

20%  (21  -  Was  difficult.  Here's  why  18:  No  scale 

24:  The  colors  indicating  the  concentrations  blend  into  the 
background.  The  lines  are  extremely  thin.  (Plume  concentrations) 
_ Was  not  relevant.  Here’s  why _ 

2.2.2.  On  the  2-plot  (Wind  and  Plume)  output. . . 

2- Plot,  Wind  Field  Plot  (left  side)  -  Finding  compass  north:  (select  one) 

80%  (81  -  Was  easy. 

10%  (11  -  Was  sort  of  easy.  Here’s  why  10:  Knowing  that  North  is  always  up. 

10%  ( 1 )  -  Was  difficult.  Here’s  why  18:  No  indicator 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  airflow  direction:  (select  one) 

100%  (101  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  speed  of  the  airflow:  (select  one) 
90%  (91- Was  easy. 

10%  (l)-Was  sort  of  easy.  Here’s  why  10:  By  going  off  the  station  &  speed  there. 

Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

II-2 
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2.3  Evaluating  the  Plume  Model: 

2.3.1  Did  you  start  the  ALOHA  Model  for  this  exercise?  Yes-1  No-7  (Circle  one) 

If  no,  please  go  to  Question  2.3.2,  10:  Done  by  ARL  Staff. 

If  yes,  please  continue. 

2.3. 1.1  Was  the  ALOHA  start  up  user-friendly?  Yes-1  No  (Circle  one) 

2.3. 1.2  Was  gathering  the  plume  model  input  difficult?  Yes.  No-1  Sort  of.  (Circle  one) 

Comments: _ 

2.3. 1.3  Was  entering  the  plume  model  input  difficult?  Yes.  No-1.  Sort  of.  (Circle  one) 
Comments:  11:  Did  not  input  any  data 

15:  Did  not  do  any  input 

2.3.2.  On  the  2-plot  (Wind  and  Plume)  output. . .  (#8  did  not  answer  2.3.2) 

2-Plot,  Plume  Plot  (right  side)  -  Finding  compass  north:  (select  one) 

80%  (8)  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

10%  (1)  -  Was  difficult.  Here’s  why  18:  No  indicator 

_ Was  not  relevant.  Here’s  why _ 

2- Plot,  Plume  Plot  (right  side)  -  Determining  hazardous  areas  generated  by  the  plume: 
(select  one) 

80%  (8)  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

10%  (1)  -  Was  difficult.  Here’s  why  18:  No  distance  scale  to  reference 
_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Plume  Plot  (right  side)-Detennining  the  most/least  hazardous  areas:  (select  one) 
80%  (8)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easv.Here’s  why  18:  Lines  very  thin,  better  when  you  filled  in  the  red  area. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 
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L-REAC™  SYSTEM  EVALUATION:  CASE  III  -  System  Functionality 


Case  HI:  No  Plume/Plume  Scenarios,  with  multiple  locations:  (30  minutes) 

•  STATUS:  Wind  Field  Model  (5m  resolution)  withNo  Plume  &  Plume  scenarios. 

•  SCENARIOS:  1622-No  plume  &  Plume;  Plume  at  user-selected  site  within  Post. 

Evaluator  #:  3,8,  10, 11,  13,  14,  15, 16,  18,  24  [  10  total  ] 

Evaluation  Date/Time:  _  _ MT 

1 .  Access  to  L-REAC™  System  Output:  (#8  and  24:  Not  answer  1.1  and  1.2  questions.) 

1.1.  Describe  your  experience  with  the  DMZ  network  connection  (L-REAC™  output  access): 

80%  (8)  -  The  DMZ  connection  ran  without  any  interruptions. 

_ The  DMZ  Connection  ran  with  interruptions,  (please  describe) _ 

_ Other: _ 

1.2  Frequency  of  output  updates: 

50%  (5)  -  The  frequency  of  output  updates  was  sufficient. 

30%  (3)  -  The  frequency  of  output  updates  needs  to  be  more  often. 

10:  Might  need  quicker  updates  in  dealing  with  a  plume. 

_ The  frequency  of  output  updates  can  be  less  often. 

_ Other _ 

2.  The  L-REAC™  System  Output: 

2. 1  During  the  evaluation  period,  which  of  the  following  was  seen  on  your  output  display? 

(Select  all  that  apply): 

1- plot  graphic: 

100%  (10)-  An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

90%  (9)  -  The  date  and  time  stamp. 

80%  (8)  -  The  height  of  the  output  winds. 

90%  (9)  -  The  wind  field  model  wind  arrows  (blue). 

90%  (9)  -  The  wind  field  model  streamlines  (yellow). 

70%  (7)  -  The  measured  wind  arrows  (red).  18:  Barely. 

80%  (8)  -  A  “No  Plume”  notice  [May  only  be  present  during  initial  pail  of  test  sequence], 
70%  (7)  -  A  plume  overlay. 

70%  (7)  -  A  plume  gradient  description  of  the  gradients  used. 

90%  (9)  -  The  WSMR  Surface  Automated  Meteorology  System  (SAMS)  data  list  (text  box). 

2- plot  graphic: 

90%  19)  -  Two  area  maps. 

100%  (10)-  Time  stamp  of  wind  data. 

100%  (10)-  The  height  of  output  winds  (Left  plot  header). 

100%  (10)-  The  wind  model  streamlines  (Left  plot). 

100%  (10)-  The  color-gradient  wind  velocities  (Left  plot). 

60%  (6)  -  A  “No  Plume”  notice  [May  only  be  present  during  initial  part  of  test  sequence], 
80%  (8)  -  A  plume  overlay  (Right  plot). 

80%  (8)  -  A  plume  gradient  description  (Right  plot). 

m-i 
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2.2  Evaluating  the  Wind  Field  Model  output:  (Questions  2.2. 1  and  2.2.2) 

2.2. 1  On  the  1-plot  (Google  Earth)  output. . . 

1-Plot  -  Finding  compass  north:  (select  one) 

90%  (9)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.  Here’s  why  18:  Same. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1-Plot  -  Determining  the  direction  for  the  airflow:  (select  one) 

100%  (10)  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

1- Plot  -  Determining  the  speed  of  the  airflow:  (select  one) 

90%  (9)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.  Here’s  why  18:  No  scale. 

_ Was  difficult.  Here’s  why  _ 

_ Was  not  relevant.  Here’s  why _ 

2.2.2.  On  the  2-plot  (Wind  and  Plume)  output. . . 

2- Plot,  Wind  Field  Plot  (left  side)  -  Finding  compass  north:  (select  one) 

80%  (8)  -  Was  easy. 

1 0%  ( 1 )  -  Was  sort  of  easy.  Here’s  why  10:  Known  North 

10%  (1)  -  Was  difficult.  Here’s  why  18:  No  Indicator 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  airflow  direction:  (select  one) 

100%  (10)  -  Was  easy. 

_ Was  sort  of  easy.  Here's  why _ 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Wind  Field  Plot  (left  side)  -  Determining  the  speed  of  the  airflow:  (select  one) 
90%  (9)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.  Here’s  why  10:  Using  color  chart  at  bottom. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

ffl-2 
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Yes-1  No-8  (Circle  one) 
10:  Input  done  by  ARL 


2.3  Evaluating  the  Plume  Model: 

2.3.1  Did  you  start  the  ALOHA  Model  for  this  exercise? 

If  no,  please  go  to  Question  2.3.2, 

If  yes,  please  continue. 

2.3. 1.1  Was  the  ALOHA  start  up  user-friendly?  Yes-1  No  (Circle  one) 

2.3. 1.2  Was  gathering  the  plume  model  input  difficult?  Yes.  No-1  Sort  of.  (Circle  one) 

Comments: _ 

2.3. 1.3  Was  entering  the  plume  model  input  difficult?  Yes.  No.  Sort  of.  (Circle  one) 
Comments:  15:  Did  not  do  this 

2.3.2.  On  the  2-plot  (Wind  and  Plume)  output. . .  (#8  and  13  did  not  answer  2.3.2) 

2-Plot,  Plume  Plot  (right  side)  -  Finding  compass  north:  (select  one) 

#8, 13  and  16:  Didn’t  answer  this  part  of  question  2.3.2. 

60%  (61  -  Was  easy. 

_ Was  sort  of  easy.  Here’s  why _ 

10%  (1)  -  Was  difficult.  Here’s  why  18:  No  indicator 

_ Was  not  relevant.  Here's  why _ 


2-Plot,  Plume  Plot  (right  side)  -  Determining  hazardous  areas  generated  by  the  plume: 
(select  one)  #8  and  13:  Didn’t  answer  question  2.3.2. 

70%  (7)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.Here’s  why  18:  Would  help  to  fill  most  hazardous  w/color. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 

2-Plot,  Plume  Plot  (right  side)-Determiningthe  most/least  hazardous  areas:  (select  one) 
#8  and  13:  Didn’t  answer  question  2.3.2. 

70%  (7)  -  Was  easy. 

10%  (1)  -  Was  sort  of  easy.  Here’s  why  18:  Should  fill  area  with  AEGL-3  with  red. 

_ Was  difficult.  Here’s  why _ 

_ Was  not  relevant.  Here’s  why _ 


ffl-3 
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Intentionally  Left  Blank. 
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Appendix  B.  Administrative  Questionnaire-Evaluator  Results 


The  Administrative  Questionnaire  focused  on  administrative  (“big  picture”)  questions  and 
applications.  The  first  two  categories  evaluated  were  the  instructions  for  accessing  the 
L-REAC™  System,  and  the  output  of  the  L-REAC™  System  as  seen  through  the  evaluator’s 
professional  requirements.  A  short  prose  (written)  section  allowed  them  to  describe  their  overall 
impression  of  the  L-REAC™  System,  where  they  thought  the  L-REAC™  product  future  vision 
should  be,  and  to  make  marketing  suggestions.  The  questionnaire  concluded  by  soliciting  “Final 
Comments.” 

There  were  nine  Administrative  Questionnaires  filled  out  and  returned;  therefore,  all  the  statistics 
were  based  on  a  total  of  nine  submissions.  (Evaluator  #24  did  not  fill  out  this  questionnaire 
form.)  Where  an  evaluator  skipped  or  missed  a  question,  a  note  was  placed  after  the  question, 
but  the  tally  was  still  based  on  nine  total  submissions. 

Note:  If  the  tallies  had  been  based  on  the  actual  number  of  evaluators  who  filled  out  the 
question,  the  tallies  would  have  been  higher  in  magnitude. 
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L-KKAC1MSV  STEM  KVAIIAIION:  Administrative  Questions 

Evaluator  #:  3,  8,  10,  It,  13,  14, 15, 16, 18  [  9  total  ]  -  #24  did  not  fill  in  this  form. 

Evaluation  Date/Time: _  _ MT 

1.  Accessing  L-REAC™  Output:  The  SOP  for  accessing  L-REAC™  output  was: 

100%  (91  -  Easy  to  follow. 

_  Easy  to  follow,  but  I'd  recommend  the  following  improvement(s): _ 

_ Difficult  to  follow.  Comment:  _ 

2.  Application: 

2.1  Briefly  describe  your  application  for  the  L-REAC™  System  output  (ie,  Fire/Rescue 
applications.  Dispatch  applications,  etc)  : 

3:  Was  Sufficient,  worked  quick. 

8:  Ops  Center  and  CBRNE. 

10:  For  all  of  the  above  plus  for  emergency  use  such  as  plume  data. 

11:  IOC 

13:  Force  protection/personnel  notification/evacuation 

14:  CBRNE  applications  for  displaying  plume  modeling  for  decision  makers. 

15:  IOC/EOC 

16:  Step  by  step  procedures  easy  to  follow 

18:  Emergency  MST  which  encompasses  Fire/Police/Dispatch  and  providing  info  for  decision  makers. 

Would  L-REAC™  help  you  to  do  your  work?  Yes-89%  (8)  No  (Circle  one). 

2.2  Which  L-REAC™  output  display  information  was  useful  for  your  application  (check  line 
next  to  all  selections  that  apply;  please  do  NOT  circle  the  “+”  until  answering  question  2.3): 


1-plot  graphic: 

(#18  circled  the  “+”) 

100%  (91  - 

+ 

An  area  of  interest  (AOI)  map  (Google  Earth  Map). 

100%  191  - 

+ 

The  data  time  stamp. 

100%  (91  - 

+ 

The  height  of  output  winds  presented. 

100%  (91- 

+ 

The  wind  field  model  wind  arrows  (blue). 

100%  (9)  - 

+ 

The  wind  field  model  streamlines  (yellow). 

100%  ("91  - 

+ 

The  measured  wind  arrows  (red). 

89%  (81  - 

+ 

A  “No  Plume'’  notice. 

100%  (91- 

+ 

A  plume  overlay. 

100%  (91  - 

+ 

A  plume  gradient  description  of  the  gradients  used. 

100%  (91  - 

+ 

The  Surface  Automated  Meteorology  System  (SAMS)  data  list  (textbox). 

2-plot  graphic: 
100%  (91 

+ 

Two  area  maps. 

100%  (91 

+ 

Time  stamp  of  wind  data. 

100%  (91 

+ 

The  height  of  output  winds  (Left  plot  header). 

100%  (91 

+ 

The  wind  model  streamlines  (Left  plot). 

100%  (91 

+ 

The  color-gradient  wind  speeds  (Left  plot). 

89%  (81 

+ 

A  “No  Plume”  notice. 

100%  (91 

+ 

A  plume  overlay  (Right  plot). 

100%  (91 

+ 

A  plume  gradient  description  (Right  plot). 
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2.3  Please  select  the  information  listed  in  question  2.2  that  was  ‘most  useful",  by  circling 
the  “  +  ”  next  to  the  1-plot  &  2-plot  line  descriptions  (multiple  selections  are  ok). 

78%  ( 7)  -  If  ALL  the  information  was  “most  useful”  then  check  this  line  only. 

2.4  Comparing  graphical  outputs  (check  the  statement  that  best  summarizes  your  observation, 
and  provide  comments):  (#16  did  not  answer  2.4) 

89%  (8)  Both  the  1-plot  and  2-plot  displays  were  equally  useful.  Comments: _ 

3:  Nice  to  see  the  different  colors;  hard  to  tell  which  colors  were,  looked  like  2  yellows. 
8:  IP  displays  all  wind  speed  info  and  2  plot  displays  the  plume  much  more  clearly. 

10:  Plotl  was  mere  usefulthen  Plot  2  when  there  was  no  plume.  But  when  there  was  a 
plume,  it  was  good  to  have  the  smaller  Google  picture  in  the  plume  box  on  Plot  2; 
and  having  the  option  to  go  to  Plot  1  for  an  expanded  view. 

14:  They  allowed  for  comparison  in  order  to  double  check  original  information. 

15:  Both  have  different  uses  and  give  useful  information 
18:  Critical  for  current  decision  making  and  future  planning. 

_ The  1-plot  output  was  more  useful  than  the  2-plot  because: _ 


The  2-plot  output  was  more  useful  than  the  1-plot  because: 


2.5  Which  would  be  better  for  a  Government  Range  (such  as  WSMR  Post),  in  your  opinion: 

(#11  did  not  select  an  option  for  2.5,  but  did  give  a  comment) 

56%  (51  Multiple  organizations  have  their  own  dedicated  L-REAC™  Systems,  or 
33%  (3)  One  organization  having  a  dedicated  system  with  multiple  organizations  seeing  the 
output. 

Briefly  explain  why: 

3:  Maybe  2  or  3  organizations,  in  the  case  that  1  organization  went  down,  no  electricity,  etc. 

8:  It  would  be  best  for  different  organizations  to  have  a  common  point  of  reference. 

10:  It  would  be  good  to  have  at  least  another  organization  with  the  system  in  case  the  initial 
L-REAC  System  is  down  or  destroyed. 

11:  Can  conceptualize  the  situation  fast. 

13:  Each  organization  would  be  able  to  adapt  for  specific  mission  requirements. 

14:  This  would  allow  for  one-source  information  dissemination  to  multiple  agencies  and 
avoid  confusion  and  distortion  of  infonnation. 

15:  Everyone  would  have  different  uses. 

16:  Better  uniformity  and  control  of  data 

18:  WSMR  IOC  operates  24/7  and  having  on  hand  availability  is  important.  No  need  for 
ARL  operators  to  arrive. 
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2.6  Presuming  the  L-REAC™  System  was  run  in  an  IOC,  to  what  groups/professionals  would 
you  recommend  the  L-REAC™  System  output  be  sent  (select  all  that  apply): 

89%  (8)  IOC  100%  (9)  Fire/Medical  67%  (6)  IC :  in  IOC  56%  (5)  Garrison  Commander 
89%  (8)  EOC  89%  (8)  Police  78%  (7)  IC:  on  location  33%  ( 31  Commanding  General 

Other/Comments : 

8:  CBRNE  as  well. 

18:  Senior  commanders  tend  to  overreact  with  too  much  info.  Best  to  keep  the  info  at  the 
responder  and  dispatcher  level.  Also,  would  like  DES  9-1-1  center  added. 

2.7  Presuming  multiple  organizations  could  have  their  own  dedicated  L-REAC™  Systems,  what 
WSMR  organizations  would  you  envision  having  their  own  L-REAC™  System  (select  all 
that  apply): 

100%  (9)  IOC  100%  (9)  Fire/Medical  44%  (4)  IC:  in  IOC  33%  (3)  Garrison  Commander 
78%  (7)  EOC  44%  141  Police  56%  (51  IC:  on  location  22%  (2)  Commanding  General 

Other/Comments:  18:  WSMR  dispatch 

2.8  What  situations/emergencies  would  best  fit  the  L-REAC™  System? 

Comments: 

3:  Bombs  smoke,  chemical  spill,  fire. 

8:  Chemical  or  biological  warfare. 

10:  Fires  on  or  around  Post.  Chemical  spills  or  fall  out  from  hazardous  test. 

11:  Gas  leak.  Chemical  spill.  Fire. 

13:  Chemical  Threat. 

14:  Any  CBRNE  event;  Hazmat  spills,  etc. 

15:  Hi  winds  and  toxic  air  situations. 

16:  Toxic  release,  fire,  smoke  or  weather  related  (tornado). 

18:  All  initial  fire  dept  callouts.  Chem./hazmat/gas  leaks.  All  FBR  incidents. 

2.9  Do  you  currently  have  a  system  similar  to  L-REAC™? 

NO- 7 8%  (7)  ->  10, 11, 13, 14, 15, 16, 18 

If  Yes,  what  is  the  name  of  the  system  you  have? _ [#8:  N/A] _ 

If  Yes,  is  L-REAC™  an  improvement  over  what  you  have?  Yes-1.  No-1.  Sort  of.  (Circle  one) 
Comments: _  [#3:  misunderstood  questions] 
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3.  OVERALL  IMPRESSION: 

3.1  What  did  you  like  about  the  system? 

3:  Real  time  data  and  visual  with  buildings  and  surrounding  area. 

8:  Clear  &  concise.  Displays  the  most  pertinent  information. 

10:  Easy  to  use  &  understand.  It  helped  with  the  initial  overview  done  by  Gail  a  week  prior. 
11:  Great  tool  with  potential 

13:  Tlie  capabilities  for  real  world  use  as  a  force  protection  tool. 

14:  It  gives  real  time  information  and  can  also  produce  a  plume  (if  necessary). 

15:  Very  easy  to  understand  and  read  winds 
16:  Clear  and  concise  output  and  (user  friendly) 

18:  Everything,  wind  flow,  speed,  current  data,  toxic  plume  data 

3.2  What  were  the  greatest  strengths? 

3:  wind  speed  and  direction.  The  probability  of  the  plume  direction.  Safe  points. 

8:  It  gives  enough  infonnation  so  that  if  someone  outside  of  the  organization  came  to  assist, 
they  should  be  able  to  assess  the  situation  relatively  quickly. 

10:  The  ability  to  add  a  plume 

11:  Ease  of  Use 

13:  The  ability  to  determine  hazards  plume  areas  for  personnel  evacuation  purposes. 

14:  Ease  of  use 

15:  The  prediction  of  the  plumes 

16:  The  amount  of  different  data  elements 

18:  Wind  speed,  direction  and  flow 

3.3  Is  the  system  providing  timely  infonnation?  Yes-89%  (81.  No.  Sort  of-11%  (11.  (Circle  one) 

Comments: _ 

3:  Good  to  have  live  data  when  presented  questions  from  the  IOC 

10:  Yes;  but  during  an  emergency  it  would  be  nice  to  have  update  quicker  when  there  was  a 
plume  involved. 

13:  Would  eventually  like  to  see  the  system  running  at  real  time. 

14:  But  I  think  that  we  could  lessen  the  update  times  (if  possible) 

18:  Yes 
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3.4  Is  the  system  providing  enough  /  too  many  details? 

Comments:  _ Enough-100%  (9).  /  Too  manv-22%  (2)  -  #8  and  10  is  ves/no 

3:  No,  as  in  “no  comments  to  add”. 

8:  Perhaps  the  type  of  chemical  and  the  N-hour  could  be  included. 

10:  Yes;  it  might  be  more  helpful  if  there  were  more  known  (RED)  arrows  in  the  area. 

11:  Enough  detail  for  now  ->  I  use  it  and  a  large  map  to  view  immediate  then  potential 
effects 

15:  Not  at  all  (as  in  ‘there  are  not  too  many  details’). 

13:  From  my  perspective  the  system  has  the  required  details. 

14:  It  is  providing  what  is  needed  for  decision  makers  to  get  timely  and  up-to-date  data  to 
deal  with  the  crisis  at  hand. 

16:  Enough  details. 

18:  Enough  details;  no  comments. 

3.5  Is  the  flexibility  (zoom  and  pan)  in  the  visualization  useful? 

Yes-66%  (6).  No-11%  (1).  Sort  of-22%  (2).  (Circle one)  Comments: _ 

3:  Focuses  in  the  area  of  interest. 

8:  This  makes  it  easier  to  be  sure  where  on  WSMR  the  incident  is  located  for  those 
unfamiliar. 

10:  We  were  not  able  to  zoom  in  &  out  during  the  cases;  but  the  ability  to  zoom  in&  out  is 
very  helpful . 

11:  Did  not  use  the  function 
15:  We  did  not  do  this 

18:  Yes,  very  useful  in  the  emergency  mgt  arena. 

4.  FUTURE  VISION:  What  improvements  to  L-REAC™  would  you  recommend? 

3:  Phone  applications,  web  access  if  evacuated  off  post. 

8:  Add  N-hour  and  the  type  of  chemicals  released. 

10:  The  ability  to  cover  a  wider  area  &  zoom  in  to  your  area  of  interest  like  on  Google 
Earth.  Off  main  cantonment  areas  like  the  F.B.R.  for  use  during  a  radiological  plume. 

11:  More  locations  providing  real-time  data  (red  arrow);  APP  creation  for  smart  phones 

13:  My  future  vision  for  LREAC  would  be  a  system  with  real  time  display  to  incorporate 
multiple  Plume  data. 

16:  Would  like  to  see  graduation  to  hand-held  that  soldier  could  use  in  the  field 
18:  1)  Ability  to  search  for  multiple  chemical  presence 

2)  Continuous/autonomous  system  monitoring/operational 

3)  Monitor  multiple  locations 
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5.  MARKETING:  Would  you  recommend  this  system  be  purchased?  Ycs-89%  (8  ). 

If  so,  why? 

3:  yes.  Useful  tool  in  determining  any  wind  plume  threats. 

8:  It  has  virtually  every  bit  of  information  first  responders  need. 

10:  It  would  be  useful  with  local  &  county  dispatch/fire  &  EOC’s. 

11:  Yes 

13:  Yes,  this  tool  would  be  helpful  for  emergency  response  and  building  evacuations  all  over 
post. 

14:  Yes,  It  provides  real-time  meteorological  information  to  the  end  user  allowing  him/her 
to  make  timely  and  important  decisions 

16:  Yes,  it  can  save  lives! 

18:  Yes,  this  tools  ability  to  determine  wind  flow  around  structures  is  very  useful  for  fire 
fighters  and  EM.  Selling  the  product  would  be  great,  too  bad  those  that  need  it  most  would 
most  likely  not  afford  it. 

6.  Any  Final  Comments: 

10:  Good  &  easy  system  to  use.  Is  this  the  system  used  by  the  weather  people  on  TV  that 
show  wind  maps  of  the  area;  or  can  they  be  tied  in  together. 

11:  Keep  the  visuals  (GUI)  clean 

14:  Keep  up  the  great  work. 

15:  I  think  the  IOC  could  use  a  bigger  map  for  the  wall,  so  that  when  things  get  hopping,  we 
don’t  have  a  crowd  behind  a  computer.  Also,  it  would  help  show  the  bigger  picture. 

16:  Great  demonstration. 

18:  Incredible  what  comes  out  of  burning  crap!  Also,  would  like  one  day  a  week  for  a 
month  or  so  to  train  operators  by  having  L-REAC  on-line  and  processing  data  requests  from 
the  IOC,  A  Wednesday  is  preferable. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ADLO 

Associate  Director  for  Laboratory  Operations 

AEGL 

acute  exposure  guideline  level 

AGL 

above  ground  level 

AOI 

area(s)  of  interest 

ARL 

U.  S.  Army  Research  Laboratory 

CBRNE 

Chemical,  Biological,  Radiological,  Nuclear  Effects 

CISD 

Computational  &  Information  Sciences  Directorate 

DoD 

Department  of  Defense 

DPTMS 

Directorate  of  Plans,  Training,  Mobilization  and  Security 

EM 

Emergency  Management 

EOC 

Emergency  Operations  Center 

EPA 

Environmental  Protection  Agency 

EUD 

End  User  Display 

FBR/F.B.R. 

Fast  Burst  Reactor 

Fire  Dept 

Fire  and  Rescue  Department 

FY11 

fiscal  year  2011 

FY12 

fiscal  year  2012 

GEMS 

Gains  in  the  Education  of  Mathematics  &  Science 

GUI 

Graphical  User  Interface 

html 

hypertext  markup  language 

IC 

Incident  Command 

IOC 

Installation  Operations  Center 

km 

kilometers 

L-REAC™ 

Local-Rapid  Evaluation  of  Atmospheric  Conditions  System 
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m 

meters 

mgt/MST 

management 

min 

minute 

NO  A  A 

National  Oceanic  and  Atmospheric  Administration 

Ops 

Operations 

OS 

operating  system 

PC 

personal  computer 

PoC 

Proof  of  Concept 

QC 

Quality  Control 

SAMS 

Surface  Automated  Meteorological  System 

SIP 

Shelter-in-Place 

SLAD 

Survivability/Lethality  Analysis  Directorate 

W03US 

White  Sands  Missile  Range  2003  Urban  Study 

W05US 

White  Sands  Missile  Range  2005  Urban  Study 

W07US 

White  Sands  Missile  Range  2007  Urban  Study 

WSMR 

White  Sands  Missile  Range 
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No.  of 

Copies  Organization 

1  (PDF)  ADMNSTR 

DEFNS  TECHL  INFO  CTR 
DTIC  OCP 

8725  JOHN  J  KINGMAN  RD  STE  0944 
FT  BELVOIR  VA  22060-6218 

3  HCs  US  ARMY  RSRCH  LAB 
ATTN  RDRL  CIO  MT 
TECHL  PUB 
ATTN  RDRL  CIO  LL 
TECHL  LIB 
ATTN  IMNE  ALC  HRR 
MAIL  &  RECORDS  MGMT 
2800  POWDER  MILL  ROAD 
ADELPHI  MD  20783-1 197 


6  CDs  US  ARMY  RSRCH  LAB 
10  HCs  G  VAUCHER 

ATTN  RDRL  CIE  D 
BLD  1622 
WSMR  NM  88002 

1  CD  US  ARMY  RSRCH  LAB 
R  BRICE 

ATTN  RDRL  CIE  D 
BLDG  1622 
WSMR  NM  88002 

1  CD  US  ARMY  RSRCH  LAB 
S  OBRIEN 
ATT  RDRL  CIE  D 
BLDG  1622 
WSMR  NM  88002 

Total  22  (1  PDF,  13  HCs,  8  CDs) 
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